Fix flaky linter clang-tidy relative path (#94093)

There are some occurrences when clang-tidy linter fails flakily with the following error, which is very weird:

```
>>> Lint for FILE:
  Error (CLANGTIDY) command-failed
    Failed due to FileNotFoundError:
    [Errno 2] No such file or directory: '.lintbin/clang-tidy'
```

For examples,

* 0a93e6db5a
* 203b2cad3e

The binary is definitely there as the log shows that it has been downloaded successfully from S3.  Looking a bit closer, I notice that the linter uses `os.chdir` to jump around between the workspace and the build folder.  And it also refers to the binary with the relative path `.lintbin/clang-tidy` which doesn't exist in the latter.  AFAIK, the current working directory is per process (https://stackoverflow.com/questions/16388400/what-is-a-thread-specific-os-chdir-and-mkdir-in-python), so I suspect that there is a race here where one thread chdir into build while another thread tries to lint another file.  Thus the fix to use the absolute path to clang-tidy

Pull Request resolved: https://github.com/pytorch/pytorch/pull/94093
Approved by: https://github.com/malfet
This commit is contained in:
Huy Do
2023-02-04 02:05:38 +00:00
committed by PyTorch MergeBot
parent e071d72f3c
commit 59a81b695a

View File

@ -253,6 +253,14 @@ def main() -> None:
abs_build_dir = Path(args.build_dir).resolve()
# Get the absolute path to clang-tidy and use this instead of the relative
# path such as .lintbin/clang-tidy. The problem here is that os.chdir is
# per process, and the linter uses it to move between the current directory
# and the build folder. And there is no .lintbin directory in the latter.
# When it happens in a race condition, the linter command will fails with
# the following no such file or directory error: '.lintbin/clang-tidy'
binary_path = os.path.abspath(args.binary)
with concurrent.futures.ThreadPoolExecutor(
max_workers=os.cpu_count(),
thread_name_prefix="Thread",
@ -261,7 +269,7 @@ def main() -> None:
executor.submit(
check_file,
filename,
args.binary,
binary_path,
abs_build_dir,
): filename
for filename in args.filenames