frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

correct filename issue for test_cpp_extensions_aot (#60604)

Browse Source 
Summary:
Using file copy to make actual ninja vs. no_ninja suffixed python test files.
This is to trick xmlrunner to report test cases in the correct folder.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/60604

Test Plan:
- CI reports correctly into the corresponding folders
- If download the test statistics, calculate shards now doesn't need custom logic to handle `test_cpp_extensions_aot`

CI result shown it is working properly:
https://github.com/pytorch/pytorch/pull/60604/checks?check_run_id=2900038654 vs
https://github.com/pytorch/pytorch/pull/60604/checks?check_run_id=2900038673

Reviewed By: albanD

Differential Revision: D29349562

Pulled By: walterddr

fbshipit-source-id: e86e6bc0db288a2a57bea3c5f8edf03be1773944
This commit is contained in:
Rong Rong (AI Infra)
2021-06-24 09:19:16 -07:00
committed by Facebook GitHub Bot
parent 9cab894367
commit 40d2fe1053
1 changed files with 10 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
test/run_test.py
View File

@ -436,6 +436,8 @@ def calculate_job_times(reports: List["Report"]) -> Dict[str, float]:
new_avg = (curr_avg * curr_count + test_file['total_seconds']) / new_count
jobs_to_times[name] = (new_avg, new_count)
# This is no longer needed after https://github.com/pytorch/pytorch/pull/60604,
# TODO remove this once viable/strict move pass the merged commit.
# if there's 'test_cpp_extensions_aot' entry in jobs_to_times, add 'test_cpp_extensions_aot_ninja'
# and 'test_cpp_extensions_aot_no_ninja' duplicate entries to ease future computation since
# test_cpp_extensions_aot_no_ninja and test_cpp_extensions_aot_ninja are Python test jobs that
@ -602,7 +604,7 @@ def test_cuda_primary_ctx(test_module, test_directory, options):
return run_test(test_module, test_directory, options, extra_unittest_args=['--subprocess'])
def _test_cpp_extensions_aot(test_module, test_directory, options, use_ninja):
def _test_cpp_extensions_aot(test_directory, options, use_ninja):
if use_ninja:
try:
cpp_extension.verify_ninja_availability()
@ -632,6 +634,9 @@ def _test_cpp_extensions_aot(test_module, test_directory, options, use_ninja):
# "install" the test modules and run tests
python_path = os.environ.get('PYTHONPATH', '')
from shutil import copyfile
test_module = 'test_cpp_extensions_aot' + ('_ninja' if use_ninja else '_no_ninja')
copyfile(test_directory + '/test_cpp_extensions_aot.py', test_directory + '/' + test_module + '.py')
try:
cpp_extensions = os.path.join(test_directory, 'cpp_extensions')
install_directory = ''
@ -646,16 +651,16 @@ def _test_cpp_extensions_aot(test_module, test_directory, options, use_ninja):
return run_test(test_module, test_directory, options)
finally:
os.environ['PYTHONPATH'] = python_path
if os.path.exists(test_directory + '/' + test_module + '.py'):
os.remove(test_directory + '/' + test_module + '.py')
def test_cpp_extensions_aot_ninja(test_module, test_directory, options):
return _test_cpp_extensions_aot('test_cpp_extensions_aot', test_directory,
options, use_ninja=True)
return _test_cpp_extensions_aot(test_directory, options, use_ninja=True)
def test_cpp_extensions_aot_no_ninja(test_module, test_directory, options):
return _test_cpp_extensions_aot('test_cpp_extensions_aot',
test_directory, options, use_ninja=False)
return _test_cpp_extensions_aot(test_directory, options, use_ninja=False)
def test_distributed(test_module, test_directory, options):