mirror of
https://github.com/pytorch/pytorch.git
synced 2025-10-20 21:14:14 +08:00
gracefully handle tokenize.TokenError
in funcname parser. Adds support for non-Python source (#148737)
This change allows defining python functions in non-python source and having them be able to compiled by torch.compile. The existing implementation already returns None for the case where the file couldn't be read, so returning None (by making an empty funcname cache) makes sense for the case of non-python source code too. Example [basilisp](https://github.com/basilisp-lang/basilisp): ```clojure (import torch) (import [torch.nn.functional :as F]) (torch/rand 10) (defn f {:decorators [torch/compile]} [x] (* (F/relu x) x)) (f (-> (torch/randn 100) (.cuda))) ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/148737 Approved by: https://github.com/williamwen42
This commit is contained in:
committed by
PyTorch MergeBot
parent
73c8068cf8
commit
abcec55532
@ -31,7 +31,7 @@ def _add_file(filename: str) -> None:
|
||||
try:
|
||||
with tokenize.open(filename) as f:
|
||||
tokens = list(tokenize.generate_tokens(f.readline))
|
||||
except OSError:
|
||||
except (OSError, tokenize.TokenError):
|
||||
cache[filename] = {}
|
||||
return
|
||||
|
||||
|
Reference in New Issue
Block a user