[torch.library] Change Library.__del__ into weakref.finalize (#101829)

`__del__` is a bit difficult to use, because when it is called, it is
not guaranteed that anything it uses has not been cleaned up.

Ed tells me he got the following exception one day, which is what
prompted this PR.
```
Exception ignored in: <function Library.__del__ at 0x7fa36d211e50>
Traceback (most recent call last):
  File "/data/users/ezyang/a/pytorch/torch/library.py", line 139, in
  __del__
  AttributeError: 'NoneType' object has no attribute 'remove'
```

One solution is to use weakref.finalize, which lets one define a
function to be run when the object is deleted that can hold references
to specific things it needs.

Another solution is to just check if the object is None, but I like the
weakref solution better.

Test Plan:
- new test
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101829
Approved by: https://github.com/ezyang

torch/library.py

@@ -2,6 +2,7 @@ from ._ops import OpOverload
 from typing import Set
 import traceback
 import torch
+import weakref
 
 __all__ = ['Library', 'impl', 'define']
 
@@ -44,9 +45,14 @@ class Library:
         filename, lineno = frame.filename, frame.lineno
         self.m = torch._C._dispatch_library(kind, ns, dispatch_key, filename, lineno)
         self.ns = ns
-        self._op_impls = set()
+        self._op_impls: Set[str] = set()
         self.kind = kind
         self.dispatch_key = dispatch_key
+        # Use a finalizer to setup the "destructor" instead of __del__.
+        # Python __del__ can lead to weird things (globals and locals may already
+        # be gone when __del__ actually gets called!). finalizers help the
+        # situation because it lets us capture references and keeps them alive
+        weakref.finalize(self, _del_library, _impls, self._op_impls)
 
     def __repr__(self):
         return "Library(kind={}, ns={}, dispatch_key={})>".format(self.kind, self.ns, self.dispatch_key)
@@ -131,13 +137,9 @@ class Library:
         _impls.add(key)
         self._op_impls.add(key)
 
-    def __del__(self):
-        # _op_impls might not have been initialized if an error was thrown in __init__
-        _op_impls_ = getattr(self, '_op_impls', None)
-        if _op_impls_:
-            for key in self._op_impls:
-                _impls.remove(key)
-            del self.m
+
+def _del_library(captured_impls, op_impls):
+    captured_impls -= op_impls
 
 
 # decorator to register python functions for library ops