|
|
6fa3592dc6
|
Dataloader benchmark script (#159432)
This script adds a simple dataloading benchmark tracking throughput and memory.
The output looks like this
```
System Information:
PyTorch version: 2.9.0a0+gitf87d117
PyTorch location: /home/divyanshkhanna/pytorch/torch/__init__.py
Torchvision version: 0.24.0a0+f52c4f1
Torchvision location: /home/divyanshkhanna/pytorch/vision/torchvision/__init__.py
CUDA available: True
CUDA device: NVIDIA PG509-210
CPU count: 192
Physical CPU cores: 96
Total system memory: 1510.11 GB
Loading dataset from imagenet/val (1 copies)
Dataset size: 50000
--- Benchmarking DataLoader with worker_method=multiprocessing ---
Memory before DataLoader creation: 500.59 MB
Detailed memory information:
USS (Unique Set Size): 499.00 MB
PSS (Proportional Set Size): 500.74 MB
RSS (Resident Set Size): 497.39 MB
Memory after DataLoader creation: 1127.61 MB
Memory increase: 627.02 MB
Starting training loop with 1 epochs (max 100 batches per epoch)
Epoch 1, Batch 10, Time: 0.2910s, Memory: 12044.50 MB
Epoch 1, Batch 20, Time: 0.2909s, Memory: 12185.71 MB
Epoch 1, Batch 30, Time: 0.2909s, Memory: 10654.93 MB
Epoch 1, Batch 40, Time: 0.2909s, Memory: 12378.26 MB
Epoch 1, Batch 50, Time: 0.2907s, Memory: 12402.28 MB
Epoch 1, Batch 60, Time: 0.2909s, Memory: 10559.35 MB
Epoch 1, Batch 70, Time: 0.2907s, Memory: 12644.69 MB
Epoch 1, Batch 80, Time: 0.2909s, Memory: 12654.65 MB
Epoch 1, Batch 90, Time: 0.2909s, Memory: 12727.20 MB
Epoch 1, Batch 100, Time: 0.2908s, Memory: 12722.09 MB
Results:
Worker method: multiprocessing
DataLoader init time: 0.1553 seconds
Average batch time: 0.3408 seconds
Samples per second: 375.53
Peak memory usage: 12738.76 MB
Memory increase: 12238.17 MB
```
> TODO: This script right now is CPU-only friendly and GPU friendly. But it might be worth upgrading it to test against a canonical DistributedDataParallel setup on say a 1x8 node. Or maybe we can keep that as a separate script inside `benchmarks`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/159432
Approved by: https://github.com/ramanishsingh
|
2025-08-06 19:05:19 +00:00 |
|
|
|
e6d8ed02cb
|
PyTorch Data Sampler benchmark (#156974)
## Motivation
Many PRs optimizing samplers (for eg https://github.com/pytorch/pytorch/pull/147706, https://github.com/pytorch/pytorch/pull/137423) are leveraging an adhoc script for benchmarking samplers. The script and outputs are often copied over in PRs. We want to begin centralizing benchmarks for torch.utils.data components.
## What ?
* This PR adds a new sub-folder in `benchmarks` for `data`. This is aimed to cover benchmarking scripts for torch.utils.data components like dataloader and sampler.
* Specifically, this PR includes a simple script to time samplers. This is often "copy-pasted" in PRs optimizing samplers. Having it in a centralized location should prevent that, and allow a common standard.
## Output
```
Benchmark Results:
+--------------+-------------+----------------+-----------+-----------+
| Batch Size | Drop Last | Original (s) | New (s) | Speedup |
+==============+=============+================+===========+===========+
| 4 | True | 0.004 | 0.0088 | -119.62% |
+--------------+-------------+----------------+-----------+-----------+
| 4 | False | 0.0083 | 0.009 | -9.23% |
+--------------+-------------+----------------+-----------+-----------+
| 8 | True | 0.003 | 0.0074 | -147.64% |
+--------------+-------------+----------------+-----------+-----------+
| 8 | False | 0.0054 | 0.0075 | -38.72% |
+--------------+-------------+----------------+-----------+-----------+
| 64 | True | 0.0021 | 0.0056 | -161.92% |
+--------------+-------------+----------------+-----------+-----------+
| 64 | False | 0.0029 | 0.0055 | -92.50% |
+--------------+-------------+----------------+-----------+-----------+
| 640 | True | 0.002 | 0.0055 | -168.75% |
+--------------+-------------+----------------+-----------+-----------+
| 640 | False | 0.0024 | 0.0062 | -161.35% |
+--------------+-------------+----------------+-----------+-----------+
| 6400 | True | 0.0021 | 0.0055 | -160.13% |
+--------------+-------------+----------------+-----------+-----------+
| 6400 | False | 0.0021 | 0.0068 | -215.46% |
+--------------+-------------+----------------+-----------+-----------+
| 64000 | True | 0.0042 | 0.0065 | -55.29% |
+--------------+-------------+----------------+-----------+-----------+
| 64000 | False | 0.0029 | 0.0077 | -169.56% |
+--------------+-------------+----------------+-----------+-----------+
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/156974
Approved by: https://github.com/ramanishsingh
|
2025-06-27 04:49:43 +00:00 |
|
