Add PyTorch Glossary (#40639)

Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/40639

Differential Revision: D22421207

Pulled By: gmagogsfm

fbshipit-source-id: 7df8bfc85e28bcf1fb08892a3671e7a9cb0dee9c

GLOSSARY.md

@@ -0,0 +1,82 @@
+# PyTorch Glossary 
+
+- [PyTorch Glossary](#pytorch-glossary)
+- [Operation and Kernel](#operation-and-kernel)
+  - [ATen](#aten)
+  - [Operation](#operation)
+  - [Native Operation](#native-operation)
+  - [Custom Operation](#custom-operation)
+  - [Kernel](#kernel)
+  - [Compound Operation](#compound-operation)
+  - [Composite Operation](#composite-operation)
+  - [Non-Leaf Operation](#non-leaf-operation)
+  - [Leaf Operation](#leaf-operation)
+  - [Device Kernel](#device-kernel)
+  - [Compound Kernel](#compound-kernel)
+- [JIT Compilation](#jit-compilation)
+  - [JIT](#jit)
+  - [TorchScript](#torchscript)
+  - [Tracing](#tracing)
+  - [Scripting](#scripting)
+
+# Operation and Kernel
+
+## ATen
+Short for "A Tensor Library". The foundational tensor and mathematical
+operation library on which all else is built.
+
+## Operation
+A unit of work. For example, the work of matrix multiplication is an operation
+called aten::matmul.
+
+## Native Operation
+An operation that comes natively with PyTorch ATen, for example aten::matmul.
+
+## Custom Operation
+An Operation that is defined by users and is usually a Compound Operation.
+For example, this
+[tutorial](https://pytorch.org/docs/stable/notes/extending.html) details how
+to create Custom Operations.
+
+## Kernel
+Implementation of a PyTorch operation, specifying what should be done when an 
+operation executes.
+
+## Compound Operation
+A Compound Operation is composed of other operations. Its kernel is usually
+device-agnostic. Normally it doesn't have its own derivative functions defined.
+Instead, AutoGrad automatically computes its derivative based on operations it
+uses.
+
+## Composite Operation
+Same as Compound Operation.
+
+## Non-Leaf Operation
+Same as Compound Operation.
+
+## Leaf Operation
+An operation that's considered a basic operation, as opposed to a Compound
+Operation. Leaf Operation always has dispatch functions defined, usually has a
+derivative function defined as well. 
+
+## Device Kernel
+Device-specific kernel of a leaf operation.
+
+## Compound Kernel
+Opposed to Device Kernels, Compound kernels are usually device-agnostic and belong to Compound Operations.
+
+# JIT Compilation
+
+## JIT
+Just-In-Time Compilation.
+
+## TorchScript
+An interface to the TorchScript JIT compiler and interpreter.
+
+## Tracing
+Using `torch.jit.trace` on a function to get an executable that can be optimized
+using just-in-time compilation.
+
+## Scripting
+Using `torch.jit.script` on a function to inspect source code and compile it as
+TorchScript code.

README.md

@@ -323,6 +323,7 @@ Three pointers to get you started:
 - [Tutorials: get you started with understanding and using PyTorch](https://pytorch.org/tutorials/)
 - [Examples: easy to understand pytorch code across all domains](https://github.com/pytorch/examples)
 - [The API Reference](https://pytorch.org/docs/)
+- [Glossary](https://github.com/pytorch/pytorch/blob/master/GLOSSARY.md)
 
 ## Resources