Lightweight at-most-once logging for API usage (#20745)

Summary:
Resubmit #20698 which got messed up.

Idea is that when PyTorch is used in a custom build environment (e.g. Facebook), it's useful to track usage of various APIs centrally. This PR introduces a simple very lightweight mechanism to do so - only first invocation of a trigger point would be logged. This is significantly more lightweight than #18235 and thus we can allow to put logging in e.g. TensorImpl.

Also adds an initial list of trigger points. Trigger points are added in such a way that no static initialization triggers them, i.e. just linking with libtorch.so will not cause any logging. Further suggestions of what to log are welcomed.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/20745

Differential Revision: D15429196

Pulled By: dzhulgakov

fbshipit-source-id: a5e41a709a65b7ebccc6b95f93854e583cf20aca
This commit is contained in:
Dmytro Dzhulgakov
2019-05-23 23:14:48 -07:00
committed by Facebook Github Bot
parent 8cde4c4d22
commit c25e33789e
29 changed files with 247 additions and 45 deletions

34
test/test_logging.py Normal file
View File

@ -0,0 +1,34 @@
import os
import sys
import subprocess
import torch
from common_utils import TestCase, run_tests
class LoggingTest(TestCase):
@staticmethod
def _runAndCaptureStderr(code):
env = os.environ.copy()
env["PYTORCH_API_USAGE_STDERR"] = "1"
pipes = subprocess.Popen(
[sys.executable, '-c', code],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env=env)
return pipes.communicate()[1].decode('ascii')
def testApiUsage(self):
"""
This test verifies that api usage logging is not triggered via static
initialization. Since it's triggered at first invocation only - we just
subprocess
"""
s = self._runAndCaptureStderr("import torch")
self.assertRegexpMatches(s, "PYTORCH_API_USAGE.*import")
# import the shared library directly - it triggers static init but doesn't call anything
s = self._runAndCaptureStderr("from ctypes import CDLL; CDLL('{}')".format(torch._C.__file__))
self.assertNotRegexpMatches(s, "PYTORCH_API_USAGE")
if __name__ == '__main__':
run_tests()