TD outside of test job (#118250)

Give TD it's own job so that each shard can get the results from this one job artifact and they will always be in sync with each other/no longer need to worry about consistently issues

* Move test discovery to its own file that is not dependent on torch so it can be run without building torch
  * Cannot do cpp test discovery before building pytorch
* Move TD calculation to own file that will create a json file with the final results
* TD is now job/build env agnostic
* TD will rank all tests, including those that test jobs may not want to run (ex it will rank distributed tests along with default tests, even though these tests are never run on the same machine together)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/118250
Approved by: https://github.com/huydhn

tools/test/test_upload_stats_lib.py

@@ -1,13 +1,18 @@
 import decimal
 import inspect
+import pathlib
+import sys
 import unittest
 from typing import Any, Dict
 from unittest import mock
 
+REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent.parent
+sys.path.insert(0, str(REPO_ROOT))
 from tools.stats.upload_metrics import add_global_metric, emit_metric
 
 from tools.stats.upload_stats_lib import BATCH_SIZE, upload_to_rockset
 
+sys.path.remove(str(REPO_ROOT))
 
 # default values
 REPO = "some/repo"
@@ -278,7 +283,7 @@ class TestUploadStats(unittest.TestCase):
         mock.patch.dict(
             "os.environ",
             {
-                "BUILD_ENVIRONMENT": "",
+                "GITHUB_JOB": "",
             },
         ).start()