[Pytorch Edge] Generic Build Features for Selective Build (#67817)

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67817

Implementation of build features as a useable feature. Includes tracing support and selectivity support. Follow up of Dhruv's prototype in D30076214.

The general idea is to allow selectivity of arbitrary sections of the codebase through the 2 apis,
BUILD_FEATURE_REQUIRED(NAME), and
BUILD_FEATURE_AVAILABLE(NAME)

References
PyTorch Edge Team Workplace group post link: https://fb.workplace.com/groups/pytorch.edge.team/posts/905584476662959/
Quip talking about some early ideas related to build features: https://fb.quip.com/iur3ApU9q29v
Google Doc about most recent discussion and details: https://docs.google.com/document/d/1533zuN_9pwpQBa4RhtstUjT5B7guowblqJz35QYWPE0/edit

Will remove the copy kernel example after. Its just here as an example.
ghstack-source-id: 142850218

Test Plan: CI, dummy traced a model, and played around with its unit test if i removed the traced value from the yaml

Reviewed By: dhruvbird

Differential Revision: D32151856

fbshipit-source-id: 33764c1f6902a025e53807b784792a83c8385984
This commit is contained in:
Jacob Szwejbka
2021-11-09 15:35:40 -08:00
committed by Facebook GitHub Bot
parent 43ef6816f2
commit 55e3b23abe
11 changed files with 184 additions and 19 deletions

View File

@ -84,6 +84,7 @@ def write_selected_mobile_ops(
) -> None:
root_ops = extract_root_operators(selective_builder)
custom_classes = selective_builder.custom_classes
build_features = selective_builder.build_features
with open(output_file_path, "wb") as out_file:
body_parts = [selected_mobile_ops_preamble]
# This condition checks if we are in selective build.
@ -93,6 +94,7 @@ def write_selected_mobile_ops(
# This condition checks if we are in tracing based selective build
if selective_builder.include_all_non_op_selectives is False:
body_parts.append("#define TORCH_CUSTOM_CLASS_ALLOWLIST " + (";".join(sorted(custom_classes))) + ";\n\n")
body_parts.append("#define TORCH_BUILD_FEATURE_ALLOWLIST " + (";".join(sorted(build_features))) + ";\n\n")
body_parts.append(get_selected_kernel_dtypes_code(selective_builder))
header_contents = "".join(body_parts)