frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

[package] Add an intern keyword (#57341)

Browse Source 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/57341

Require that users be explicit about what they are going to be
interning. There are a lot of changes that are enabled by this. The new
overall scheme is:

PackageExporter maintains a dependency graph. Users can add to it,
either explicitly (by issuing a `save_*` call) or explicitly (through
dependency resolution). Users can also specify what action to take when
PackageExporter encounters a module (deny, intern, mock, extern).

Nothing (except pickles, tho that can be changed with a small amount
of work) is written to the zip archive until we are finalizing the
package. At that point, we consult the dependency graph and write out
the package exactly as it tells us to.

This accomplishes two things:
1. We can gather up *all* packaging errors instead of showing them one at a time.
2. We require that users be explicit about what's going in packages, which is a common request.

Differential Revision: D28114185

Test Plan: Imported from OSS

Reviewed By: SplitInfinity

Pulled By: suo

fbshipit-source-id: fa1abf1c26be42b14c7e7cf3403ecf336ad4fc12
This commit is contained in:
Michael Suo
2021-05-12 16:21:26 -07:00
committed by Facebook GitHub Bot
parent d230045fde
commit 01d0eb9dac
12 changed files with 455 additions and 173 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
test/package/test_misc.py
View File

@ -69,6 +69,7 @@ class TestMisc(PackageTestCase):
import package_a.subpackage
obj = package_a.subpackage.PackageASubpackageObject()
he.intern("**")
he.save_module(module_a.__name__)
he.save_module(package_a.__name__)
he.save_pickle("obj", "obj.pkl", obj)
@ -106,6 +107,7 @@ class TestMisc(PackageTestCase):
with PackageExporter(buffer, verbose=False) as he:
import package_a.subpackage
he.intern("**")
obj = package_a.subpackage.PackageASubpackageObject()
he.save_pickle("obj", "obj.pkl", obj)
@ -124,6 +126,7 @@ class TestMisc(PackageTestCase):
obj = package_a.subpackage.PackageASubpackageObject()
with PackageExporter(buffer, verbose=False) as pe:
pe.intern("**")
pe.save_pickle("obj", "obj.pkl", obj)
buffer.seek(0)
@ -145,6 +148,7 @@ class TestMisc(PackageTestCase):
obj = package_a.subpackage.PackageASubpackageObject()
with PackageExporter(buffer, verbose=False) as pe:
pe.intern("**")
pe.save_pickle("obj", "obj.pkl", obj)
buffer.seek(0)
@ -168,6 +172,7 @@ class TestMisc(PackageTestCase):
obj = package_a.subpackage.PackageASubpackageObject()
with PackageExporter(buffer, verbose=False) as pe:
pe.intern("**")
pe.save_pickle("obj", "obj.pkl", obj)
buffer.seek(0)
@ -188,6 +193,7 @@ class TestMisc(PackageTestCase):
buffer = BytesIO()
with PackageExporter(buffer, verbose=False) as pe:
pe.intern("**")
pe.save_module(mod.__name__)
buffer.seek(0)