5ad7611b52
Reland vision pinned commit hash update ( #164492 )
...
Redo https://github.com/pytorch/pytorch/pull/154694
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164492
Approved by: https://github.com/yangw-dev
2025-10-12 04:53:27 +00:00
90b4e130d6
[Benchmark] cleanup torchbench models ( #164816 )
...
Prune models from TorchInductor dashboard to reduce ci cost. This PR prunes torchbench models according to the [doc](https://docs.google.com/document/d/1nLPNNAU-_M9Clx9FMrJ1ycdPxe-xRA54olPnsFzdpoU/edit?tab=t.0 ), which removes timm and huggingface models from torchbench.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164816
Approved by: https://github.com/anijain2305 , https://github.com/seemethere , https://github.com/huydhn , https://github.com/malfet
2025-10-09 00:31:25 +00:00
83458197d1
[Benchmark] remove old timm models from benchmark ( #164805 )
...
Prune models from TorchInductor dashboard to reduce ci cost. This PR prunes for timm models according to the [doc](https://docs.google.com/document/d/1nLPNNAU-_M9Clx9FMrJ1ycdPxe-xRA54olPnsFzdpoU/edit?tab=t.0 ), which reduces from 60 to 14 models.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164805
Approved by: https://github.com/anijain2305 , https://github.com/seemethere , https://github.com/huydhn , https://github.com/malfet
2025-10-08 17:14:58 +00:00
1927783aa3
Revert "Reland vision pinned commit hash update ( #164492 )"
...
This reverts commit 6861a270624b44954826688f8dad668eb0154452.
Reverted https://github.com/pytorch/pytorch/pull/164492 on behalf of https://github.com/izaitsevfb due to see autorevert msg above, inductor breakage is legit ([comment](https://github.com/pytorch/pytorch/pull/164492#issuecomment-3379537888 ))
2025-10-08 04:38:26 +00:00
6861a27062
Reland vision pinned commit hash update ( #164492 )
...
Redo https://github.com/pytorch/pytorch/pull/154694
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164492
Approved by: https://github.com/yangw-dev
2025-10-07 22:45:05 +00:00
412c6d28ec
[ROCm][CI] additional dynamo benchmarks for inductor-periodic ( #164279 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/164279
Approved by: https://github.com/jeffdaily
Co-authored-by: Jeff Daily <jeff.daily@amd.com >
2025-10-04 00:55:17 +00:00
0319556a35
Revert "[vision hash update] update the pinned vision hash ( #154694 )"
...
This reverts commit bcafea5c92ca2ee1b0dc8f6d8b62ecabb6f40228.
Reverted https://github.com/pytorch/pytorch/pull/154694 on behalf of https://github.com/yangw-dev due to break the unittest for inductor with improved, update benchmarks/dynamo/ci_expected_accuracy/inductor_torchbench_inference.csv, see failure example https://github.com/pytorch/pytorch/actions/runs/18185852421/job/51776537817 ([comment](https://github.com/pytorch/pytorch/pull/154694#issuecomment-3362285901 ))
2025-10-02 17:32:04 +00:00
bcafea5c92
[vision hash update] update the pinned vision hash ( #154694 )
...
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml ).
Update the pinned vision hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/154694
Approved by: https://github.com/pytorchbot
Co-authored-by: Huy Do <huydhn@gmail.com >
2025-10-02 07:02:40 +00:00
dad54ca7c0
Add mistral/gpt-oss to benchmarks ( #163565 )
...
Potential issues
* gpt-oss-20b is probably too big (I can't run on my devserver)
* Mistral requires HF authentication
* Mistral also takes a while to run the performance checks (need to wait for CI)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163565
Approved by: https://github.com/huydhn
2025-09-24 06:12:36 +00:00
972140b7e9
[benchmark] Add HF LLM benchmarks ( #156967 )
...
Results in https://docs.google.com/spreadsheets/d/1xXOPg9JjEmPx0zc5QBNdyXQq8-K2_r4ybHaiS-q7pZ0/edit?gid=88695043#gid=88695043
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156967
Approved by: https://github.com/huydhn
Co-authored-by: Huy Do <huydhn@gmail.com >
2025-09-14 07:41:06 +00:00
07a4e9fea8
[benchmarks] Skip mobilenetv3_large_100 in CI for accuracy ( #161570 )
...
To keep the CI green - https://github.com/pytorch/pytorch/issues/161419
Its unclear if this is a real failure. And debugging it is non trivial.
Skipping for now to keep the CI greenst
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161570
Approved by: https://github.com/BoyuanFeng , https://github.com/zou3519
2025-08-27 03:44:04 +00:00
01bcf9a40d
Bump transformers pin ( #159291 )
...
Trying to update hf pin.
Benchmarking run to figure out issues
<img width="1356" height="123" alt="image" src="https://github.com/user-attachments/assets/fbc435f3-a7cb-4280-9636-2ea6d15d7b6d " />
Retrying - https://github.com/pytorch/pytorch/pull/156118
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159291
Approved by: https://github.com/BoyuanFeng , https://github.com/huydhn
Co-authored-by: Huy Do <huydhn@gmail.com >
2025-08-12 05:14:17 +00:00
2068235c0a
Add timm_efficientnet to flaky models after cuda 12.6 update in CI/CD ( #148788 )
...
After https://github.com/pytorch/pytorch/pull/148612
This model have become flaky
Tracking this regression in an issue : https://github.com/pytorch/pytorch/issues/148699
Pull Request resolved: https://github.com/pytorch/pytorch/pull/148788
Approved by: https://github.com/izaitsevfb , https://github.com/malfet
2025-03-10 13:40:41 +00:00
754fb834db
[BE][CI] bump ruff
to 0.9.0: string quote styles ( #144569 )
...
Reference: https://docs.astral.sh/ruff/formatter/#f-string-formatting
- Change the outer quotes to double quotes for nested f-strings
```diff
- f'{", ".join(args)}'
+ f"{', '.join(args)}"
```
- Change the inner quotes to double quotes for triple f-strings
```diff
string = """
- {', '.join(args)}
+ {", ".join(args)}
"""
```
- Join implicitly concatenated strings
```diff
- string = "short string " "short string " f"{var}"
+ string = f"short string short string {var}"
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144569
Approved by: https://github.com/Skylion007
ghstack dependencies: #146509
2025-02-24 19:56:09 +00:00
5fd15a04b7
[ROCm] Enable inductor-periodic testing for MI300 ( #144594 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144594
Approved by: https://github.com/malfet , https://github.com/huydhn
Co-authored-by: Jeff Daily <jeff.daily@amd.com >
2025-02-10 17:42:09 +00:00
498a7808ff
Fix unused Python variables outside torch/ and test/ ( #136359 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/136359
Approved by: https://github.com/albanD
2024-12-11 17:10:23 +00:00
d3e932dc10
[CI] Add inductor cpu accuracy test running on AVX2 runners ( #128682 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128682
Approved by: https://github.com/jgong5 , https://github.com/desertfire
2024-07-26 13:24:41 +00:00
a676b7c5f3
Add XGLMForCausalLM to the flaky model list ( #129776 )
...
Not failing on devGPU. Went to CI machine ... flaky. So adding to the flaky list.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/129776
Approved by: https://github.com/mlazos
ghstack dependencies: #129583 , #129610 , #129775
2024-06-29 05:47:28 +00:00
9c77332116
[torch.compile][ci] Flaky models in CI (similar to DISABLED_TEST) ( #128715 )
...
These models are really flaky. I went into the CI machine and ran the model many times, sometime it fails, sometimes it passes. Even Pytorch-eager results change from run to run, so the accuracy comparison is fundamentally broken/non-deterministic. I am hitting these issues more frequently in inlining work. There is nothing wrong with inlining, I think these models are on the edge of already-broken accuracy measurement, and inlining is just pushing it in more broken direction.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128715
Approved by: https://github.com/eellison
2024-06-14 20:17:03 +00:00
a595a50653
[CI] Use expected accuracy csv files to check benchmark test status ( #98839 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/98839
Approved by: https://github.com/ezyang
2023-04-15 13:54:41 +00:00