[pytorch] Add logger for pt2 compile chromium events to hive (#139941)

Summary:
X-link: https://github.com/pytorch/benchmark/pull/2535

Logging raw chromium events to hive per job run enables us to build combined rank perfetto traces without having to depend on Logarithm and deal with things like rate limits etc.

We can easily build a utility to query hive and upload traces to manifold and view them on perfetto

Test Plan:
Launch a job

```
buck2 run mode/opt //aps_models/examples/dlrm:dlrm_train_app -- --config-name train_mast_fsdp_torchdynamo launcher.data_project=apf_ai_infra launcher.fbl_entitlement=ai_infra_training_rnd_tc  launcher.hardware=TC_ANY_80G
```

Local run
```
Perfetto: ['https://interncache-all.fbcdn.net/manifold/perfetto-artifacts/tree/ui/index.html?url=https://interncache-all.fbcdn.net/manifold/pt2_compile_traces_test/tree/pt2_trace_files/aps-ppanchalia-426838c277/0/0/2bc9975d-921c-4766-9cb2-e7ce9833ae96.json']
```

{F1954710538}

Differential Revision: D65525513

Pull Request resolved: https://github.com/pytorch/pytorch/pull/139941
Approved by: https://github.com/jamesjwu
This commit is contained in:
Prajesh Praveen Anchalia
2024-11-14 18:27:38 +00:00
committed by PyTorch MergeBot
parent 50ab68fa22
commit 9ff368c270
2 changed files with 8 additions and 0 deletions

View File

@ -76,6 +76,7 @@ from torch._subclasses.meta_utils import is_sparse_compressed
from torch._utils_internal import (
log_chromium_event_internal,
log_compilation_event,
record_chromium_event_internal,
signpost_event,
)
from torch.fx._utils import _format_graph_code, lazy_format_graph_code
@ -1177,6 +1178,7 @@ class ChromiumEventLogger:
suppress_context=False,
expect_trace_id=False, # Not every chromium event will have a trace_id
)
record_chromium_event_internal(event)
return event
def log_instant_event(

View File

@ -266,3 +266,9 @@ def log_chromium_event_internal(
start_time_ns: int,
):
return None
def record_chromium_event_internal(
event: Dict[str, Any],
):
return None