[profiler] add send/recv src/dst info (#111811)

Summary: There is an ask to add src/dst to nccl trace. This feels like the easiest way to do - adding it to metadata seems to require plumbing a few stacks so will be more work

Test Plan: {F1128545195}

Reviewed By: davidberard98

Differential Revision: D50560692

Pull Request resolved: https://github.com/pytorch/pytorch/pull/111811
Approved by: https://github.com/davidberard98, https://github.com/aaronenyeshi, https://github.com/fduwjj

torch/testing/_internal/distributed/distributed_test.py

@@ -185,13 +185,17 @@ DEFAULT_TIMEOUT = 300
 CUSTOMIZED_TIMEOUT = {"test_DistributedDataParallel": 500}
 
 
-def get_profiling_event(postfix, profiler):
+def get_profiling_event(event_name, profiler):
     event_list = (
         profiler.events()
         if isinstance(profiler, torch.profiler.profile)
         else profiler.function_events
     )
-    return [event for event in event_list if event.name.endswith(postfix)]
+    return [
+        event for event in event_list if (
+            event.name.endswith(event_name) or event.name.startswith(event_name)
+        )
+    ]
 
 
 # Base error message substring on unfinished reductions.