Manuel Candales
aea57b3aa3
AOTI MPS Shim Implementation (#163865)
## MPS Shim API
* Updated MPS shimification API with handles and function declarations:
* `AOTIMetalShaderLibraryHandle` and `AOTIMetalKernelFunctionHandle` types
* Library management: `aoti_torch_mps_create_shader_library`, `aoti_torch_mps_delete_shader_library`, `aoti_torch_mps_get_kernel_function`
* Kernel execution: `aoti_torch_mps_run_command_block`, `aoti_torch_mps_start_encoding`, `aoti_torch_mps_dispatch` variants, etc
## MPS Shader Codegen
* Modified to generate source constants instead of direct `DynamicMetalShaderLibrary` instantiation:
* **Before**: `at::native::mps::DynamicMetalShaderLibrary mps_lib_0(R"MTL(...)MTL");`
* **After**: `const char* mps_lib_0_source = R"MTL(...)MTL";`
* Updated kernel call generation to use shimified functions:
* Generates calls to shimified API instead of direct libtorch calls
## Before vs After Comparison
### Section 1: Shader Library
**Before (Direct Library Object)**
```cpp
at::native::mps::DynamicMetalShaderLibrary mps_lib_0(R"MTL(
...
)MTL");
```
**After (Source String)**
```cpp
const char* mps_lib_0_source = (R"MTL(
...
)MTL");
```
### Section 2: Getter Functions & RAII Management
**Before (Direct Library Access)**
```cpp
const std::shared_ptr<at::native::mps::MetalKernelFunction> get_mps_lib_0() {
static const auto func = mps_lib_0.getKernelFunction("generated_kernel");
return func;
}
AOTIMetalKernelFunctionHandle get_mps_lib_0_handle() {
static const auto handle = AOTIMetalKernelFunctionHandle(get_mps_lib_0().get());
return handle;
}
```
**After (Shim API + RAII Wrapper)**
```cpp
AOTIMetalKernelFunctionHandle get_mps_lib_0_handle() {
static auto kernel_handle = []() {
AOTIMetalShaderLibraryHandle lib_handle = nullptr;
AOTIMetalKernelFunctionHandle kern_handle = nullptr;
aoti_torch_mps_create_shader_library(mps_lib_0_source, &lib_handle);
aoti_torch_mps_get_kernel_function(lib_handle, "generated_kernel", &kern_handle);
// RAII wrapper with custom deleter
auto lib_deleter = [](AOTIMetalShaderLibraryHandle h) {{
if (h) aoti_torch_mps_delete_shader_library(h);
}};
using LibDeleter = decltype(lib_deleter);
using LibPtr = std::unique_ptr<AOTIMetalShaderLibraryOpaque, LibDeleter>;
// Return pair of kernel handle and library smart pointer for cleanup
return std::make_pair(kern_handle, LibPtr(lib_handle, lib_deleter));
}();
return kernel_handle.first;
}
```
### Section 3: Runtime Execution
**Before (Direct Library Methods)**
```cpp
void AOTInductorModel::run_impl(...) {
...
get_mps_lib_0()->runCommandBlock([&] {
get_mps_lib_0()->startEncoding();
aoti_torch_mps_set_arg_tensor(get_mps_lib_0_handle(), 0, buf0);
aoti_torch_mps_set_arg_tensor(get_mps_lib_0_handle(), 1, arg0_1);
aoti_torch_mps_set_arg_tensor(get_mps_lib_0_handle(), 2, arg1_1);
get_mps_lib_0()->dispatch({static_cast<uint64_t>(10LL)});
});
...
} // AOTInductorModel::run_impl
```
**After (Shim API with Lambda Pattern)**
```cpp
void AOTInductorModel::run_impl(...) {
...
auto mps_lib_0_lambda_0 = [&](AOTIMetalKernelFunctionHandle handle) {
aoti_torch_mps_start_encoding(handle);
aoti_torch_mps_set_arg_tensor(handle, 0, buf0);
aoti_torch_mps_set_arg_tensor(handle, 1, arg0_1);
aoti_torch_mps_set_arg_tensor(handle, 2, arg1_1);
aoti_torch_mps_dispatch_single(handle, static_cast<uint64_t>(10LL));
};
std::function<void(AOTIMetalKernelFunctionHandle)> mps_lib_0_func_wrapper_0 = mps_lib_0_lambda_0;
aoti_torch_mps_run_command_block(get_mps_lib_0_handle(), aoti_torch_mps_shared_callback, &mps_lib_0_func_wrapper_0);
...
} // AOTInductorModel::run_impl
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/163865
Approved by: https://github.com/angelayi, https://github.com/desertfire
2025-10-09 16:06:36 +00:00
..
2025-10-05 19:32:21 +00:00
2024-12-13 22:13:12 +00:00
2025-07-17 12:08:33 +00:00
2025-06-21 18:33:38 +00:00
2025-10-09 15:42:51 +00:00
2025-06-24 04:53:54 +00:00
2025-09-30 12:27:11 +00:00
2025-03-29 01:39:13 +00:00
2025-07-25 02:37:30 +00:00
2025-10-09 15:31:23 +00:00
2025-09-17 06:42:27 +00:00
2025-10-09 04:40:54 +00:00
2025-10-01 14:10:44 +00:00
2025-10-01 21:32:59 +00:00
2025-10-09 12:43:18 +00:00
2025-10-09 03:25:15 +00:00
2025-10-09 15:42:51 +00:00
2025-10-09 09:50:59 +00:00
2025-10-08 14:23:38 +00:00
2025-10-08 14:23:38 +00:00
2025-10-09 16:06:36 +00:00
2025-09-10 04:39:20 +00:00
2025-01-27 18:12:39 +00:00
2025-10-08 20:23:13 +00:00
2025-03-29 01:39:13 +00:00
2025-07-29 03:26:09 +00:00
2025-10-05 19:32:21 +00:00
2025-10-06 17:48:50 +00:00
2025-10-05 19:32:21 +00:00
2025-09-25 06:58:58 +00:00
2025-08-06 02:26:10 +00:00
2025-10-09 03:24:50 +00:00
2025-10-05 19:32:21 +00:00
2025-07-29 03:26:09 +00:00
2025-02-22 03:44:53 +00:00
2025-10-05 19:32:21 +00:00
2025-09-02 16:53:55 +00:00
2025-09-19 07:37:14 +00:00
2025-10-09 15:42:51 +00:00
2025-08-03 20:53:58 +00:00
2025-10-02 22:22:04 +00:00
2024-09-06 08:18:38 +00:00
2024-11-04 18:30:29 +00:00
2025-10-09 11:54:10 +00:00
2025-05-12 18:30:52 +00:00
2025-10-06 12:57:29 +00:00
2025-08-08 17:41:22 +00:00
2025-09-18 16:08:13 +00:00
2025-04-26 18:10:58 +00:00
2025-02-07 06:06:18 +00:00
2025-07-30 19:30:55 +00:00
2025-10-08 20:23:13 +00:00
2025-09-16 18:28:50 +00:00
2025-06-04 14:38:13 +00:00
2025-10-08 07:27:17 +00:00
2024-11-22 20:54:55 +00:00
2025-10-08 07:27:17 +00:00
2025-09-09 15:49:21 +00:00
2025-08-04 20:37:39 +00:00
2025-09-29 15:15:10 +00:00
2025-09-29 17:50:12 +00:00
2025-04-10 21:02:14 +00:00
2025-04-25 20:15:04 +00:00
2025-01-04 14:17:20 +00:00
2025-09-09 15:49:21 +00:00
2025-10-09 03:24:50 +00:00
2025-09-23 22:15:10 +00:00
2025-01-04 10:47:51 +00:00
2024-12-18 23:02:30 +00:00
2025-10-08 18:48:45 +00:00
2025-10-08 09:09:16 +00:00
2025-09-10 07:05:14 +00:00
2025-09-23 05:02:06 +00:00
2025-10-09 09:50:59 +00:00
2025-06-14 03:37:38 +00:00
2025-09-17 16:39:11 +00:00
2025-10-09 13:09:06 +00:00
2025-08-04 20:37:39 +00:00
2025-01-23 00:31:39 +00:00
2025-09-26 15:45:02 +00:00
2024-12-18 23:02:30 +00:00
2025-09-25 17:14:19 +00:00
2025-10-06 22:42:01 +00:00
2024-12-18 23:02:30 +00:00
2025-02-08 00:55:20 +00:00
2025-01-04 10:47:51 +00:00
2025-07-09 11:02:23 +00:00
2025-07-09 11:02:23 +00:00
2025-08-13 05:50:15 +00:00
2025-07-09 11:02:23 +00:00
2025-07-10 06:34:46 +00:00
2025-10-05 19:32:21 +00:00
2025-02-28 00:47:03 +00:00
2025-06-04 14:38:13 +00:00
2025-10-05 19:32:21 +00:00
2025-10-05 19:32:21 +00:00
2025-10-02 22:22:04 +00:00
2025-10-02 22:22:04 +00:00
2025-10-02 22:22:04 +00:00
2025-10-02 22:22:04 +00:00
2024-12-18 23:02:30 +00:00
2025-10-09 03:24:50 +00:00
2025-10-05 19:32:21 +00:00
2024-12-18 23:02:30 +00:00
2025-09-26 18:26:56 +00:00
2025-07-09 11:02:23 +00:00
2025-07-09 11:02:23 +00:00
2025-10-07 22:36:25 +00:00
2025-07-09 11:02:23 +00:00
2025-07-21 21:44:49 +00:00
2025-10-07 22:36:25 +00:00
2025-09-17 20:29:12 +00:00
2024-12-18 23:02:30 +00:00
2025-01-04 10:47:51 +00:00
2025-07-17 08:57:34 +00:00
2024-12-18 23:02:30 +00:00
2025-09-10 14:19:34 +00:00
2024-12-18 23:02:30 +00:00
2025-09-12 15:02:40 +00:00
2024-12-18 23:02:30 +00:00
2025-10-03 00:48:38 +00:00
2025-08-15 00:11:55 +00:00
2025-06-12 14:42:32 +00:00
2025-09-26 17:41:00 +00:00
2025-01-04 10:47:51 +00:00
2024-12-06 21:45:18 +00:00
2025-10-02 18:46:27 +00:00
2025-10-08 15:26:50 +00:00
2025-01-04 10:47:51 +00:00
2025-09-25 13:47:46 +00:00
2025-09-26 18:26:56 +00:00
2025-10-07 00:34:14 +00:00
2025-10-08 09:09:16 +00:00
2024-12-18 23:02:30 +00:00
2025-03-18 16:09:39 +00:00
2025-07-09 11:02:23 +00:00
2025-09-29 09:08:04 +00:00
2025-09-16 12:07:50 +00:00
2024-12-27 07:58:44 +00:00
2025-09-13 07:52:50 +00:00
2025-07-09 11:02:23 +00:00
2025-09-19 19:41:33 +00:00
2025-10-01 21:32:59 +00:00
2025-10-08 18:42:37 +00:00
2025-10-05 19:32:21 +00:00
2025-10-07 20:51:22 +00:00
2025-10-08 01:46:45 +00:00
2025-09-05 20:15:29 +00:00
2025-07-11 03:21:47 +00:00
2025-10-05 19:32:21 +00:00
2025-07-17 01:27:44 +00:00
2025-10-09 12:43:18 +00:00
2025-09-15 06:50:00 +00:00
2025-08-10 18:35:42 +00:00
2025-10-05 19:32:21 +00:00
2025-10-05 19:32:21 +00:00
2025-02-25 03:47:40 +00:00
2025-08-14 17:06:27 +00:00
2025-10-06 22:42:01 +00:00
2025-09-23 07:52:00 +00:00
2025-10-08 21:41:42 +00:00
2025-06-04 01:58:52 +00:00
2025-07-09 11:02:23 +00:00
2025-01-04 10:47:51 +00:00
2024-12-18 23:02:30 +00:00
2025-09-29 01:42:01 +00:00
2025-10-05 19:32:21 +00:00
2025-09-12 16:26:54 +00:00
2025-07-09 11:02:23 +00:00
2025-09-29 01:42:01 +00:00
2025-01-26 03:37:20 +00:00
2025-10-03 00:09:47 +00:00
2025-10-06 16:20:44 +00:00
2025-09-26 23:50:09 +00:00
2025-10-08 07:27:17 +00:00
2025-10-08 07:27:17 +00:00
2025-07-20 23:49:18 +00:00
2025-07-09 11:02:23 +00:00
2025-09-15 05:44:15 +00:00
2025-02-05 19:40:10 +00:00
2024-12-12 01:18:34 +00:00
2025-10-04 15:25:45 +00:00
2025-08-16 00:54:32 +00:00
2024-12-18 23:02:30 +00:00
2025-07-09 11:02:23 +00:00
2025-02-04 19:07:04 +00:00
2025-09-16 18:00:22 +00:00