[inductor] Fix issue with set_linter, improve linter framework (#144620)

### `set_linter` only

* Fix gnarly [bug](dbed747aae/tools/test/set_linter_testdata/python_code.py.txt.python (L42)) which would have garbled Python files involving sets contained in sets.
* Better handling of new Python3.12 token types

### Both linters.

* Recover from and report on unparseable Python files
* Remove `ParseError.check()` (it made it harder to read the code)
* FileLinter is now generic on `PythonFile`

### Notes

As I started working on new docstring features, I found a nasty bug and an edge case bug in set linter, and realized both the linters crash when there is a badly-formed Python file in the repo.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/144620
Approved by: https://github.com/amjames, https://github.com/jansel
This commit is contained in:
Tom Ritchford
2025-03-12 21:36:34 +00:00
committed by PyTorch MergeBot
parent f4bffb7461
commit d90f9e9a34
11 changed files with 308 additions and 289 deletions

View File

@ -38,7 +38,7 @@ def _is_def(t: TokenInfo) -> bool:
return t.type == token.NAME and t.string in ("class", "def")
class DocstringLinter(_linter.FileLinter):
class DocstringLinter(_linter.FileLinter[_linter.PythonFile]):
linter_name = "docstring_linter"
description = DESCRIPTION
is_fixer = False
@ -87,7 +87,7 @@ class DocstringLinter(_linter.FileLinter):
for i in range(start, len(tokens)):
if tokens[i].type == token_type:
return i
_linter.ParseError.check(False, tokens[-1], error)
raise _linter.ParseError(tokens[-1], error)
for i in defs:
name = next_token(i + 1, token.NAME, "Definition with no name")