[pytorch] Add logger for pt2 compile chromium events to hive (#139941)

Summary:
X-link: https://github.com/pytorch/benchmark/pull/2535

Logging raw chromium events to hive per job run enables us to build combined rank perfetto traces without having to depend on Logarithm and deal with things like rate limits etc.

We can easily build a utility to query hive and upload traces to manifold and view them on perfetto

Test Plan:
Launch a job

```
buck2 run mode/opt //aps_models/examples/dlrm:dlrm_train_app -- --config-name train_mast_fsdp_torchdynamo launcher.data_project=apf_ai_infra launcher.fbl_entitlement=ai_infra_training_rnd_tc  launcher.hardware=TC_ANY_80G
```

Local run
```
Perfetto: ['https://interncache-all.fbcdn.net/manifold/perfetto-artifacts/tree/ui/index.html?url=https://interncache-all.fbcdn.net/manifold/pt2_compile_traces_test/tree/pt2_trace_files/aps-ppanchalia-426838c277/0/0/2bc9975d-921c-4766-9cb2-e7ce9833ae96.json']
```

{F1954710538}

Differential Revision: D65525513

Pull Request resolved: https://github.com/pytorch/pytorch/pull/139941
Approved by: https://github.com/jamesjwu
This commit is contained in:
Prajesh Praveen Anchalia
2024-11-14 18:27:38 +00:00
committed by PyTorch MergeBot
parent 50ab68fa22
commit 9ff368c270
2 changed files with 8 additions and 0 deletions

View File

@ -266,3 +266,9 @@ def log_chromium_event_internal(
start_time_ns: int,
):
return None
def record_chromium_event_internal(
event: Dict[str, Any],
):
return None