frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update Vulkan runner in benchmark binary to handle non-tensor inputs (#66123)

Browse Source 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66123

Some models may take in a list of tensors as inputs, thus the bundled inputs will contain `IValues` that are of the type `c10::List`. For Vulkan models, every tensor in the `IValue` list has to be converted to a vulkan tensor first, and this case is not currently handled by the Vulkan model wrapper in the benchmark binary.

This diff introduces `IValue` type checking to the input processor of the Vulkan model wrapper, and adds support for Tensor and List types.

Test Plan:
```
# Build the binary
cd ~/fbsource
buck build -c ndk.custom_libcxx=false -c pt.enable_qpl=0 //xplat/caffe2:ptmobile_compareAndroid\#android-arm64 --show-output
# Push it to the device
adb push buck-out/gen/xplat/caffe2/ptmobile_compareAndroid\#android-arm64 /data/local/tmp/compare_models

# Run the benchmark binary
BENCH_CMD="/data/local/tmp/compare_models"
BENCH_CMD+=" --model=$PATH_TO_MODEL"
BENCH_CMD+=" --refmodel=$PATH_TO_REFERENCE_MODEL"
BENCH_CMD+=" --input_type=float --input_dims=$MODEL_INPUT_SIZE"
BENCH_CMD+=" --iter=100"
BENCH_CMD+=" --tolerance 1e-5"
```

Reviewed By: beback4u

Differential Revision: D31276862

fbshipit-source-id: 1d9abf958963da6ecad641202f0458402bee5ced
This commit is contained in:
Stephen Jia
2021-10-05 07:58:30 -07:00
committed by Facebook GitHub Bot
parent 2a5116e159
commit df475aa1dc
1 changed files with 21 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
binaries/speed_benchmark_torch.cc
View File

@ -184,8 +184,28 @@ class vkRunner final : public Runner<T> {
inputs_.clear();
inputs_.reserve(inputs.size());
for (const auto& input : inputs) {
if (input.isTensor()) {
inputs_.emplace_back(input.toTensor().vulkan());
}
else if (input.isList()) {
const c10::List<c10::IValue> input_as_list = input.toList();
c10::List<at::Tensor> input_vk_list;
input_vk_list.reserve(input_as_list.size());
for (int i=0; i < input_as_list.size(); ++i) {
const c10::IValue element = input_as_list.get(i);
if (element.isTensor()) {
input_vk_list.emplace_back(element.toTensor().vulkan());
}
else {
CAFFE_THROW("Input of type c10::List must only contain Tensors!");
}
}
inputs_.emplace_back(c10::IValue(input_vk_list));
}
else {
CAFFE_THROW("Inputs must only contain IValues of type c10::Tensor or c10::List!");
}
}
// Run, and download the output tensor to system memory.
return module.forward(inputs_).toTensor().cpu();