7 Commits

Author SHA1 Message Date
46419c7899 Revert "[Relandx2] Rewrite the guts of torch::jit::Lexer to speed it up (#152372)"
This reverts commit 7ce6f632142b65849fa33f325c90a24bace2c130.

Reverted https://github.com/pytorch/pytorch/pull/152372 on behalf of https://github.com/malfet due to Looks like it broke distributed this time around, see f05d3e5019/1 ([comment](https://github.com/pytorch/pytorch/pull/152372#issuecomment-2837426497))
2025-04-29 04:37:40 +00:00
7ce6f63214 [Relandx2] Rewrite the guts of torch::jit::Lexer to speed it up (#152372)
Reapplying with fix for linux-manylinux-2_28-py3-cpu-s390x / build
failure
(https://github.com/pytorch/pytorch/actions/runs/14716285820/job/41300304223#logs),
which is to just update a pair of static_assert constants I got wrong.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152372
Approved by: https://github.com/wdvr, https://github.com/malfet
2025-04-28 23:55:48 +00:00
e7c19f4f69 Revert "Reapply "Rewrite the guts of torch::jit::Lexer to speed it up (#151850)" (#152250)"
This reverts commit e407ea1e5e22a41d14ce141295bf391cd46f2677.

Reverted https://github.com/pytorch/pytorch/pull/152250 on behalf of https://github.com/malfet due to Breaks s390, may be time to move build back to opt-in 2667cb69d9/1 ([comment](https://github.com/pytorch/pytorch/pull/152250#issuecomment-2836833030))
2025-04-28 22:05:12 +00:00
e407ea1e5e Reapply "Rewrite the guts of torch::jit::Lexer to speed it up (#151850)" (#152250)
Almost-exact reapply of #151850 (adding minor reviewer nits) . AFAICT it was reverted unnecessarily.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152250
Approved by: https://github.com/Skylion007, https://github.com/cyyever
2025-04-28 19:33:40 +00:00
fa1b4ef649 Revert "Rewrite the guts of torch::jit::Lexer to speed it up (#151850)"
This reverts commit 47d34261e06e2416e7a1e7d51a3d428e4ea51f9d.

Reverted https://github.com/pytorch/pytorch/pull/151850 on behalf of https://github.com/ZainRizvi due to This codev PR is breaking  on it's internal counterpart diff D73129443.  For codev PRs like this one, please always make sure the internal diff is green and then land the diff internally. The Github PR will be automatically merged ([comment](https://github.com/pytorch/pytorch/pull/151850#issuecomment-2831686141))
2025-04-26 00:44:11 +00:00
47d34261e0 Rewrite the guts of torch::jit::Lexer to speed it up (#151850)
The trie-based approach was, apparently, not efficient. This incidentally fixes a bug where "not inp" and "is note" were lexed incorrectly; see test_lexer.cpp update.

Differential Revision: [D73129443](https://our.internmc.facebook.com/intern/diff/D73129443/)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/151850
Approved by: https://github.com/Skylion007
ghstack dependencies: #151801, #151802, #151803, #151804, #151805, #151806, #151807, #151810, #151849
2025-04-25 23:49:35 +00:00
cf101d66ee Add simple direct C++ tests for torch::jit::Lexer (#151849)
We have test_jit.py, but given that I'm working on
significant changes to the lexer, it seems nice to have direct C++
tests. (Also, writing the tests caught a pair of related bugs; see the
two tests with "Bug" in their name. The rewrite will fix them.)

Differential Revision: [D73402367](https://our.internmc.facebook.com/intern/diff/D73402367/)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/151849
Approved by: https://github.com/malfet
ghstack dependencies: #151801, #151802, #151803, #151804, #151805, #151806, #151807, #151810
2025-04-25 22:39:49 +00:00