frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-11-11 22:34:53 +08:00
Code Issues Packages Projects Releases Wiki Activity

refresh numbers after accepted less than noise regression (#140029)

Browse Source 
https://github.com/pytorch/pytorch/pull/138363 regressed some benchmarks but less than noise level updating values to avoid flakiness.
<img width="803" alt="Screenshot 2024-11-07 at 10 31 29 AM" src="https://github.com/user-attachments/assets/31326452-a6ad-44b8-b324-25e953355fcf">

PASS: benchmark ('add_loop_eager', 'compile_time_instruction_count') pass, actual result 3073605220 +1.21% is within expected 3037000000 ±1.50%

PASS: benchmark ('add_loop_eager_dynamic', 'compile_time_instruction_count') pass, actual result 5700849667 +1.37% is within expected 5624000000 ±2.50%

Pull Request resolved: https://github.com/pytorch/pytorch/pull/140029
Approved by: https://github.com/bobrenjc93
This commit is contained in:
Laith Sakka
2024-11-07 10:36:40 -08:00
committed by PyTorch MergeBot
parent 83e36a6bfa
commit d1a45800a3
1 changed files with 17 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
benchmarks/dynamo/pr_time_benchmarks/expected_results.csv
View File

@ -1,65 +1,65 @@
add_loop_eager,compile_time_instruction_count,3037000000,0.015
add_loop_eager,compile_time_instruction_count,3073000000,0.015
add_loop_eager_dynamic,compile_time_instruction_count,5624000000,0.025
add_loop_eager_dynamic,compile_time_instruction_count,5700000000,0.025
add_loop_inductor,compile_time_instruction_count,24600000000,0.015
add_loop_inductor,compile_time_instruction_count,24580000000,0.015
add_loop_inductor_dynamic_gpu,compile_time_instruction_count,40740000000,0.025
add_loop_inductor_dynamic_gpu,compile_time_instruction_count,40810000000,0.025
add_loop_inductor_gpu,compile_time_instruction_count,23330000000,0.015
add_loop_inductor_gpu,compile_time_instruction_count,23290000000,0.015
basic_modules_ListOfLinears_eager,compile_time_instruction_count,1032000000,0.015
basic_modules_ListOfLinears_eager,compile_time_instruction_count,1037000000,0.015
basic_modules_ListOfLinears_inductor,compile_time_instruction_count,19150000000,0.015
basic_modules_ListOfLinears_inductor,compile_time_instruction_count,19200000000,0.015
basic_modules_ListOfLinears_inductor_gpu_force_shape_pad,compile_time_instruction_count,15770000000,0.015
basic_modules_ListOfLinears_inductor_gpu_force_shape_pad,compile_time_instruction_count,15820000000,0.015
basic_modules_ListOfLinears_inductor_gpu,compile_time_instruction_count,16060000000,0.2
basic_modules_ListOfLinears_inductor_gpu,compile_time_instruction_count,16890000000,0.2
update_hint_regression,compile_time_instruction_count,1750000000,0.02
update_hint_regression,compile_time_instruction_count,1757000000,0.02
sum_floordiv_regression,compile_time_instruction_count,1160000000,0.015
sum_floordiv_regression,compile_time_instruction_count,1171000000,0.015
symint_sum,compile_time_instruction_count,3319000000,0.015
symint_sum,compile_time_instruction_count,3321000000,0.015
aotdispatcher_inference_nosubclass_cpu,compile_time_instruction_count,2000000000,0.015
aotdispatcher_inference_nosubclass_cpu,compile_time_instruction_count,2014000000,0.015
aotdispatcher_inference_subclass_cpu,compile_time_instruction_count,5779000000,0.015
aotdispatcher_inference_subclass_cpu,compile_time_instruction_count,5826000000,0.015
aotdispatcher_partitioner_cpu,compile_time_instruction_count,9010000000,0.015
aotdispatcher_partitioner_cpu,compile_time_instruction_count,9022000000,0.015
aotdispatcher_training_nosubclass_cpu,compile_time_instruction_count,3824000000,0.015
aotdispatcher_training_nosubclass_cpu,compile_time_instruction_count,3848000000,0.015
aotdispatcher_training_subclass_cpu,compile_time_instruction_count,10270000000,0.015
aotdispatcher_training_subclass_cpu,compile_time_instruction_count,10330000000,0.015

1 add_loop_eager compile_time_instruction_count 3037000000 3073000000 0.015
2 add_loop_eager_dynamic compile_time_instruction_count 5624000000 5700000000 0.025
3 add_loop_inductor compile_time_instruction_count 24600000000 24580000000 0.015
4 add_loop_inductor_dynamic_gpu compile_time_instruction_count 40740000000 40810000000 0.025
5 add_loop_inductor_gpu compile_time_instruction_count 23330000000 23290000000 0.015
6 basic_modules_ListOfLinears_eager compile_time_instruction_count 1032000000 1037000000 0.015
7 basic_modules_ListOfLinears_inductor compile_time_instruction_count 19150000000 19200000000 0.015
8 basic_modules_ListOfLinears_inductor_gpu_force_shape_pad compile_time_instruction_count 15770000000 15820000000 0.015
9 basic_modules_ListOfLinears_inductor_gpu compile_time_instruction_count 16060000000 16890000000 0.2
10 update_hint_regression compile_time_instruction_count 1750000000 1757000000 0.02
11 sum_floordiv_regression compile_time_instruction_count 1160000000 1171000000 0.015
12 symint_sum compile_time_instruction_count 3319000000 3321000000 0.015
13 aotdispatcher_inference_nosubclass_cpu compile_time_instruction_count 2000000000 2014000000 0.015
14 aotdispatcher_inference_subclass_cpu compile_time_instruction_count 5779000000 5826000000 0.015
15 aotdispatcher_partitioner_cpu compile_time_instruction_count 9010000000 9022000000 0.015
16 aotdispatcher_training_nosubclass_cpu compile_time_instruction_count 3824000000 3848000000 0.015
17 aotdispatcher_training_subclass_cpu compile_time_instruction_count 10270000000 10330000000 0.015
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65