Commit Graph

201 Commits

Author SHA1 Message Date
20b18a58f1 Update compiler warning about ABI compatibility (#34472)
Summary:
3ac42677633a39c588c3fea19d2d4121f114edb3 already forces pytorch to use gcc>=5 everywhere
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34472

Differential Revision: D20345134

Pulled By: ezyang

fbshipit-source-id: 3ce706405e8784cac5c314500466b5f988ad31bf
2020-03-10 08:12:07 -07:00
616beb1412 [ROCm] Added support for pytorch extensions to use HIP (#32669)
Summary:
This pull request has changes for:
1. Enabling a torch module with HIP code to be compiled by cpp_extensions.py
2. Fixes for hipify module to be able to be used by a torch extension

cc: ezyang iotamudelta jeffdaily
Pull Request resolved: https://github.com/pytorch/pytorch/pull/32669

Differential Revision: D20033893

Pulled By: zou3519

fbshipit-source-id: fd6ddc8cdcd3930f41008636bb2bc9dd26cdb008
2020-02-21 12:10:02 -08:00
ffe327f7d9 Revert "Disable flaky test TestCppExtensionAOT.test_cuda_extension in… (#33404)
Summary:
… Windows CI (https://github.com/pytorch/pytorch/issues/33282)"

This reverts commit 5b922918d023126ad1f468c68577c9b599ad202d.

Fixes https://github.com/pytorch/pytorch/issues/33270.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/33404

Differential Revision: D19972594

Pulled By: ezyang

fbshipit-source-id: c8f67536fd6e4b7135171d621ad671b1b2a21fd4
2020-02-20 09:08:29 -08:00
44af8ee6cd Add pybind11 exception translator (#30588)
Summary:
Closes https://github.com/pytorch/pytorch/issues/30027

The idea here is that you can bind a function with `pybind11` in a single line and without modifying the function:
```cpp
m.def("foo", foo, py::call_guard<torch::PyWarningHandler>());
```
Where warnings are handled by the [`call_guard`](https://pybind11.readthedocs.io/en/stable/advanced/functions.html#call-guard) and exceptions are handled by the `pybind11` exception translator. To do this, I have added support for handling C++ exceptions in `torch::PyWarningHandler`'s destructor without setting the python error state before hand.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30588

Differential Revision: D19905626

Pulled By: albanD

fbshipit-source-id: 90c0a5e298b123cc0c8ab9c52c91be4e96ea47c6
2020-02-18 11:33:29 -08:00
28c5213a97 Add mechanism to pass a number of workers to cpp extensions (#33346)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/33346

Fixes #33091

This PR lets users control the number of workers that cpp extensions
uses through the environment variable `MAX_JOBS`. If the environment
variable is a non-negative integer we use that many threads; otherwise,
ninja falls back to the default.

I chose to use the name `MAX_JOBS` because we use it in PyTorch already
to control the number of workers PyTorch builds with. There is a risk
that users of cpp extensions already have `MAX_JOBS` set but we are
hoping that that risk is small and/or it means semantically the same
thing.

Test Plan: - tested locally

Differential Revision: D19911645

Pulled By: zou3519

fbshipit-source-id: d20ed42de4f845499ed38f1a1c73e9ccb620f780
2020-02-18 06:48:11 -08:00
769abddfa3 Build ahead-of-time C++ extensions with ninja on windows
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/33084

Differential Revision: D19817361

Pulled By: ezyang

fbshipit-source-id: 95a6d0ffa9beb6885c8a41688621b33da51706ae
2020-02-11 17:50:09 -08:00
6209412647 Add option to use ninja to compile ahead-of-time cpp_extensions (#32495)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/32495

Background
------------------------------
Previously, ninja was used to compile+link inline cpp_extensions and
ahead-of-time cpp_extensions were compiled with distutils. This PR adds
the ability to compile (but not link) ahead-of-time cpp_extensions with ninja.

The main motivation for this is to speed up cpp_extension builds: distutils
does not make use of parallelism. With this PR, using the new option, on my machine,
- torchvision compilation goes from 3m43s to 49s
- nestedtensor compilation goes from 2m0s to 28s.

User-facing changes
------------------------------

I added a `use_ninja` flag to BuildExtension. This defaults to
`True`. When `use_ninja` is True:
- it will attempt to use ninja.
- If we cannot use ninja, then this throws a warning and falls back to
distutils.
- Situations we cannot use ninja: Windows (NYI, I'll open a new issue
for this), if ninja cannot be found on the system.

Implementation Details
------------------------------

This PR makes this change in two steps. Please me know if it would be
easier to review this if I split this up into a stacked diff.
Those changes are:
1) refactor _write_ninja_file to separate the policy (what compiler flags
to pass) from the mechanism (how to write the ninja file and do compilation).
2) call _write_ninja_file and _run_ninja_build while building
ahead-of-time cpp_extensions. These are only used to compile objects;
distutils still handles the linking.

Change 1: refactor _write_ninja_file to seperate policy from mechanism
- I split _write_ninja_file into: _write_ninja_file and
_write_ninja_file_to_build_library
- I renamed _build_extension_module to _run_ninja_build

Change 2: Call _write_ninja_file while building ahead-of-time
cpp_extensions
- _write_ninja_file_and_compile_objects calls _write_ninja_file to only
build object files.
- We monkey-patch distutils.CCompiler.compile to call
_write_ninja_files_and_compile_objects
- distutils still handles the linking step. The linking step is not a
bottleneck so it was not a concern.
- This change only works on unix-based systems. Our code for windows
goes down a different codepath and I did not want to mess with that.
- If a system does not support ninja, we raise a warning and fall back
to the original compilation path.

Test Plan
------------------------------

Adhoc testing
- I built torchvision using pytorch master and printed out the build
commands. Next, I used this branch to build torchvision and looked at
the ninja file. I compared the ninja file with the build commands and
asserted that they were functionally the same.
- I repeated the above for pytorch/nestedtensor.

PyTorch test suite
- I split `test_cpp_extensions` into `test_cpp_extensions_aot` and
`test_cpp_extensions_jit`. The AOT (ahead-of-time) version tests
ahead-of-time and the JIT version tests just-in-time (not to be confused
with TorchScript)
- `test_cpp_extensions_aot` gets run TWICE by run_test.py, once with
a module that was built with ninja, and once with a module that was
built without ninja.
- run_test.py asserts that when we are building with use_ninja=True,
ninja is actually available on the system.

Test Plan: Imported from OSS

Differential Revision: D19730432

Pulled By: zou3519

fbshipit-source-id: 819590d01cf65e8da5a1e8019b8b3084792fee90
2020-02-05 18:49:29 -08:00
1e5aead35b Make cuda search process of cpp extension quiet (#32620)
Summary:
Fixes https://discuss.pytorch.org/t/error-with-cpp-extentions/67559.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/32620

Differential Revision: D19576164

Pulled By: soumith

fbshipit-source-id: 076229322375774bec03ef2632fc233000c15391
2020-01-26 20:26:43 -08:00
f326045b37 Fix typos, via a Levenshtein-type corrector (#31523)
Summary:
Should be non-semantic.

Uses https://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/For_machines to find likely typos, with https://github.com/bwignall/typochecker to help automate the checking.

Uses an updated version of the tool used in https://github.com/pytorch/pytorch/pull/30606 .
Pull Request resolved: https://github.com/pytorch/pytorch/pull/31523

Differential Revision: D19216749

Pulled By: mrshenli

fbshipit-source-id: 7fd489cb9a77cd7e4950c1046f925d57524960ea
2020-01-17 16:03:19 -08:00
8614860210 Uniformly apply Windows logic in cpp_extensions everywhere (#31161)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/31161

Previously, it wasn't necessary to specify `DT_NEEDED` in C++ extensions on Linux (aka pass `-l` flags) because all of the symbols would have already been loaded with `RTLD_GLOBAL`, so there wouldn't be any undefined symbols.  But when we switch to loading `_C` with `RTLD_LOCAL`, it's now necessary for all the C++ extensions to know what libraries to link with. The resulting code is clearer and more uniform, so it's wins all around.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Test Plan: Imported from OSS

Differential Revision: D19262578

Pulled By: ezyang

fbshipit-source-id: a893cc96f2e9aad1c064a6de4f7ccf79257dec3f
2020-01-09 07:28:11 -08:00
9c9d3cd550 Revert D19262570: Fix race condition when creating build dir
Test Plan: revert-hammer

Differential Revision:
D19262570

Original commit changeset: bb18c72e4264

fbshipit-source-id: 40675ef6ef4c98629deaaef0b25956f92534ff50
2020-01-03 11:17:42 -08:00
8c425dd201 Fix race condition when creating build dir (#30956)
Summary:
The original `check-and-act` style can raise `FileExistsError` when multiple processes are jit-compiling the extension on the same node.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30956

Differential Revision: D19262570

Pulled By: ezyang

fbshipit-source-id: bb18c72e42648770b47f9378ac7c3929c3c03efc
2020-01-03 07:58:26 -08:00
9305f44854 Remove BUILD_NAMEDTENSOR from codegen and .cu files (#31047)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/31047

Changelist:
- remove BUILD_NAMEDTENSOR from .cu files
- remove BUILD_NAMEDTENSOR special handling in function_wrapper.py
- remove BUILD_NAMEDTENSOR from cpp_extension.py. This code actually
did nothing because we always compile with BUILD_NAMEDTENSOR.

Test Plan: - run tests

Differential Revision: D18908442

Pulled By: zou3519

fbshipit-source-id: b239e24de58580adaf3cef573350773a38b1e4f0
2019-12-11 08:49:56 -08:00
38986e1dea Split libtorch.so back into libtorch_{cpu,cuda,hip} (#30315)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30315

The new structure is that libtorch_cpu contains the bulk of our
code, and libtorch depends on libtorch_cpu and libtorch_cuda.
This is a reland of https://github.com/pytorch/pytorch/pull/29731 but
I've extracted all of the prep work into separate PRs which can be
landed before this one.

Some things of note:

* torch/csrc/cuda/nccl.cpp was added to the wrong list of SRCS, now fixed (this didn't matter before because previously they were all in the same library)
* The dummy file for libtorch was brought back from the dead; it was previously deleted in #20774
In an initial version of the patch, I forgot to make torch_cuda explicitly depend on torch_cpu. This lead to some very odd errors, most notably "bin/blob_test: hidden symbol `_ZNK6google8protobuf5Arena17OnArenaAllocationEPKSt9type_infom' in lib/libprotobuf.a(arena.cc.o) is referenced by DSO"
* A number of places in Android/iOS builds have to add torch_cuda explicitly as a library, as they do not have transitive dependency calculation working correctly
* I had to torch_cpu/torch_cuda caffe2_interface_library so that they get whole-archived linked into torch when you statically link. And I had to do this in an *exported* fashion because torch needs to depend on torch_cpu_library. In the end I exported everything and removed the redefinition in the Caffe2Config.cmake. However, I am not too sure why the old code did it in this way in the first place; however, it doesn't seem to have broken anything to switch it this way.
* There's some uses of `__HIP_PLATFORM_HCC__` still in `torch_cpu` code, so I had to apply it to that library too (UGH). This manifests as a failer when trying to run the CUDA fuser. This doesn't really matter substantively right now because we still in-place HIPify, but it would be good to fix eventually. This was a bit difficult to debug because of an unrelated HIP bug, see https://github.com/ROCm-Developer-Tools/HIP/issues/1706

Fixes #27215 (as our libraries are smaller), and executes on
part of the plan in #29235.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Test Plan: Imported from OSS

Differential Revision: D18790941

Pulled By: ezyang

fbshipit-source-id: 01296f6089d3de5e8365251b490c51e694f2d6c7
2019-12-04 08:04:57 -08:00
bc2e6d10fa Back out "Revert D17908478: Switch PyTorch/Caffe2 to C++14"
Summary: Original commit changeset: 775d2e29be0b

Test Plan: CI

Reviewed By: mruberry

Differential Revision: D18775520

fbshipit-source-id: a350b3f86b66d97241f208786ee67e9a51172eac
2019-12-03 14:33:43 -08:00
a2ed50c920 Revert D17908478: Switch PyTorch/Caffe2 to C++14
Test Plan: revert-hammer

Differential Revision:
D17908478

Original commit changeset: 6e340024591e

fbshipit-source-id: 775d2e29be0bc3a0db64f164c8960c44d4877d5d
2019-11-27 14:57:05 -08:00
d0acc9c085 Switch PyTorch/Caffe2 to C++14 (#30406)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30406

ghstack-source-id: 94642238

Test Plan: waitforsandcastle

Differential Revision: D17908478

fbshipit-source-id: 6e340024591ec2c69521668022999df4a33b4ddb
2019-11-27 10:47:31 -08:00
352731bd6e Revert D18632773: Split libtorch.so back into libtorch_{cpu,cuda,hip}
Test Plan: revert-hammer

Differential Revision:
D18632773

Original commit changeset: ea717c81e0d7

fbshipit-source-id: 18601439f9f81c9f389020e5a0e4e04adb21772d
2019-11-21 15:01:09 -08:00
ec30d9028a Split libtorch.so back into libtorch_{cpu,cuda,hip} (#29731)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/29731

The new structure is that libtorch_cpu contains the bulk of our
code, and libtorch depends on libtorch_cpu and libtorch_cuda.

Some subtleties about the patch:
- There were a few functions that crossed CPU-CUDA boundary without API macros. I just added them, easy enough. An inverse situation was aten/src/THC/THCTensorRandom.cu where we weren't supposed to put API macros directly in a cpp file.
- DispatchStub wasn't getting all of its symbols related to static members on DispatchStub exported properly. I tried a few fixes but in the end I just moved everyone off using DispatchStub to dispatch CUDA/HIP (so they just use normal dispatch for those cases.) Additionally, there were some mistakes where people incorrectly were failing to actually import the declaration of the dispatch stub, so added includes for those cases.
- torch/csrc/cuda/nccl.cpp was added to the wrong list of SRCS, now fixed (this didn't matter before because previously they were all in the same library)
- The dummy file for libtorch was brought back from the dead; it was previously deleted in #20774
- In an initial version of the patch, I forgot to make torch_cuda explicitly depend on torch_cpu. This lead to some very odd errors, most notably "bin/blob_test: hidden symbol `_ZNK6google8protobuf5Arena17OnArenaAllocationEPKSt9type_infom' in lib/l
ibprotobuf.a(arena.cc.o) is referenced by DSO"
- A number of places in Android/iOS builds have to add torch_cuda explicitly as a library, as they do not have transitive dependency calculation working correctly. This situation also happens with custom C++ extensions.
- There's a ROCm compiler bug where extern "C" on functions is not respected. There's a little workaround to handle this.
- Because I was too lazy to check if HIPify was converting TORCH_CUDA_API into TORCH_HIP_API, I just made it so HIP build also triggers the TORCH_CUDA_API macro. Eventually, we should translate and keep the nature of TORCH_CUDA_API constant in all cases.

Fixes #27215 (as our libraries are smaller), and executes on
part of the plan in #29235.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Test Plan: Imported from OSS

Differential Revision: D18632773

Pulled By: ezyang

fbshipit-source-id: ea717c81e0d7554ede1dc404108603455a81da82
2019-11-21 11:27:33 -08:00
c0104a1c89 Fix typo in comment in cpp_extension (#30028)
Summary:
From https://github.com/pytorch/pytorch/issues/26614
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30028

Differential Revision: D18597666

Pulled By: albanD

fbshipit-source-id: 93bf0e4ee34a63df4b544d44f630a9c0fc95fd83
2019-11-20 07:16:48 -08:00
0ff1696c75 add pybind version of HANDLE_TH_ERRORS
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/26614

Test Plan: Imported from OSS

Differential Revision: D18249634

Pulled By: albanD

fbshipit-source-id: 25503f368926e0f3633c5af0f222c9bb4729f342
2019-11-07 08:35:11 -08:00
92c63d90e8 Remove support for old architectures in cpp_extension and CMake (#24442)
Summary:
This is a follow-up to gh-23408.  No longer supported are any arches < 3.5 (numbers + 'Fermi' and 'Kepler+Tegra').
Pull Request resolved: https://github.com/pytorch/pytorch/pull/24442

Differential Revision: D16889283

Pulled By: ezyang

fbshipit-source-id: 3c0c35d51b7ac7642d1be7ab4b0f260ac93b60c9
2019-08-19 06:23:33 -07:00
a3b8607811 Fix test_jit_cuda_archflags failure on py27 due to changing dict order. (#24501)
Summary:
See gh-23408.

Was failing for `pytorch_linux_xenial_cuda9_cudnn7_py2_test`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/24501

Differential Revision: D16860932

Pulled By: soumith

fbshipit-source-id: 715858d905f74a23e42a9a1da97f036a3e30f0c9
2019-08-16 12:44:16 -07:00
cd20773701 Set CUDA arch correctly when building with torch.utils.cpp_extension (#23408)
Summary:
The old behavior was to always use `sm_30`. The new behavior is:

- For building via a setup.py, check if `'arch'` is in `extra_compile_args`.  If so, don't change anything.
- If `TORCH_CUDA_ARCH_LIST` is set, respect that (can be 1 or more arches)
- Otherwise, query device capability and use that.

To test this, for example on a machine with `torch` installed for py37:
```
$ git clone https://github.com/pytorch/extension-cpp.git
$ cd extension-cpp/cuda
$ python setup.py install
$ cuobjdump --list-elf build/lib.linux-x86_64-3.7/lltm_cuda.cpython-37m-x86_64-linux-gnu.so

ELF file    1: lltm.1.sm_61.cubin
```

Existing tests in `test_cpp_extension.py` for `load_inline` and for compiling via `setup.py` in test/cpp_extensions/ cover this.

Closes gh-18657

EDIT: some more tests:

```
from torch.utils.cpp_extension import load

lltm = load(name='lltm', sources=['lltm_cuda.cpp', 'lltm_cuda_kernel.cu'])
```

```
# with TORCH_CUDA_ARCH_LIST undefined or an empty string
$ cuobjdump --list-elf /tmp/torch_extensions/lltm/lltm.so
ELF file    1: lltm.1.sm_61.cubin

# with TORCH_CUDA_ARCH_LIST = "3.5 5.2 6.0 6.1 7.0+PTX"
$ cuobjdump --list-elf build/lib.linux-x86_64-3.7/lltm_cuda.cpython-37m-x86_64-linux-gnu.so
ELF file    1: lltm_cuda.cpython-37m-x86_64-linux-gnu.1.sm_35.cubin
ELF file    2: lltm_cuda.cpython-37m-x86_64-linux-gnu.2.sm_52.cubin
ELF file    3: lltm_cuda.cpython-37m-x86_64-linux-gnu.3.sm_60.cubin
ELF file    4: lltm_cuda.cpython-37m-x86_64-linux-gnu.4.sm_61.cubin
ELF file    5: lltm_cuda.cpython-37m-x86_64-linux-gnu.5.sm_70.cubin
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23408

Differential Revision: D16784110

Pulled By: soumith

fbshipit-source-id: 69ba09e235e4f906b959fd20322c69303240ee7e
2019-08-15 15:25:15 -07:00
81e46d4f78 Fix build issue. CUDA may be installed in $CUDA_HOME/lib on macOS. (#23491)
Summary:
Closes gh-16955.
Closes https://github.com/pytorch/vision/issues/977

On Linux both `lib64` and `lib` may be present (symlinked). The reports
seem to all be about macOS, but it seems like this is also possibly more
robust on Linux and can't hurt. So not treating platforms differently.

Note that Eigen has a similar check in its CMake:

```
if(CUDA_64_BIT_DEVICE_CODE AND (EXISTS "${CUDA_TOOLKIT_ROOT_DIR}/lib64"))
  link_directories("${CUDA_TOOLKIT_ROOT_DIR}/lib64")
else()
  link_directories("${CUDA_TOOLKIT_ROOT_DIR}/lib")
endif()
 ```

There may be other issues for building from source on macOS, can't test.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23491

Differential Revision: D16538973

Pulled By: soumith

fbshipit-source-id: cc309347b7d16e718e06878d3824d0a6e40b1019
2019-07-29 08:08:43 -07:00
54c280863c Add some compiler flags for building cpp extensions on Windows (#23472)
Summary:
(1) Add `COMMON_MSVC_FLAGS` to the flags in the ninja codepath
(2) Add `/EHsc` to `COMMON_MSVC_FLAG`
(3) Remove `-fPIC` and `-std=c++11` from the flags in the windows codepath
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23472

Differential Revision: D16532993

Pulled By: soumith

fbshipit-source-id: bc2d983f5f8b4eae9c7385bf170f155679e92e87
2019-07-28 20:33:18 -07:00
34f53564b4 Don't warn when using conda compilers with utils.cpp_extension (#23396)
Summary:
The conda compiler are gcc/c++ 7.3.0, but have custom version strings
for clarity:

    x86_64-conda_cos6-linux-gnu-cc
    x86_64-conda_cos6-linux-gnu-c++

Using these compilers to build a C++ or CUDA extension now gives this warning (unnecessarily):

```
                               !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (/home/rgommers/anaconda3/envs/pytorch-nightly/bin/x86_64-conda_cos6-linux-gnu-c++) is not compatible with the compiler Pytorch was
built with for this platform, which is g++ on linux.
...
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23396

Differential Revision: D16500637

Pulled By: soumith

fbshipit-source-id: 5b2fc3593e22e9a7d07dc2c0456dbb4934ffddb2
2019-07-26 10:17:14 -07:00
0ea8e61f03 For consistent CUDA_HOME behavior (#22845)
Summary:
Align the behavior of `torch.utils.cpp_extension.CUDA_HOME` with that of `tools.setup_helpers.cuda.CUDA_HOME`.

Typically, I swapped the position of guess 2 and guess 3 in `torch.utils.cpp_extension.CUDA_HOME` .

Fixing issue https://github.com/pytorch/pytorch/issues/22844
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22845

Differential Revision: D16276241

Pulled By: zou3519

fbshipit-source-id: 3b62b439b2f794a6f3637a5fee58991f430985fe
2019-07-16 09:55:56 -07:00
e2216ada65 Properly formats errors rising up from C++ extension compilation (#22445)
Summary:
Here's a C++ extension with a missing semicolon:
```python
torch.utils.cpp_extension.load_inline('test', 'int main() { return 0 }')
```
which currently generates this error
```
RuntimeError: Error building extension 'test_v6': b'[1/2] c++ -MMD -MF main.o.d -
DTORCH_EXTENSION_NAME=test_v6 -DTORCH_API_INCLUDE_EXTENSION_H -isystem
/opt/conda/lib/python3.7/site-packages/torch/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/TH -isystem /opt/conda/lib/python3.7/site-packages/torch/include/THC
-isystem /opt/conda/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c
/tmp/torch_extensions/test/main.cpp -o main.o\nFAILED: main.o \nc++ -MMD -MF main.o.d -
DTORCH_EXTENSION_NAME=test_v6 -DTORCH_API_INCLUDE_EXTENSION_H -isystem
/opt/conda/lib/python3.7/site-packages/torch/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/TH -isystem /opt/conda/lib/python3.7/site-packages/torch/include/THC
 -isystem /opt/conda/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c
/tmp/torch_extensions/test/main.cpp -o main.o\n/tmp/torch_extensions/test/main.cpp: In
function \xe2\x80\x98int main()\xe2\x80\x99:\n/tmp/torch_extensions/test/main.cpp:2:23:
error: expected \xe2\x80\x98;\xe2\x80\x99 before \xe2\x80\x98}\xe2\x80\x99 token\n int
main() { return 0 }\n                       ^\nninja: build stopped: subcommand failed.\n'
```

After this PR, the error is
```
RuntimeError: Error building extension 'test': [1/2] c++ -MMD -MF main.o.d -
DTORCH_EXTENSION_NAME=test -DTORCH_API_INCLUDE_EXTENSION_H -isystem
/opt/conda/lib/python3.7/site-packages/torch/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/TH -isystem /opt/conda/lib/python3.7/site-packages/torch/include/THC
 -isystem /opt/conda/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c
/tmp/torch_extensions/test/main.cpp -o main.o
FAILED: main.o
c++ -MMD -MF main.o.d -DTORCH_EXTENSION_NAME=test -
DTORCH_API_INCLUDE_EXTENSION_H -isystem /opt/conda/lib/python3.7/site-
packages/torch/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.7/site-
packages/torch/include/TH -isystem /opt/conda/lib/python3.7/site-packages/torch/include/THC
 -isystem /opt/conda/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++11 -c
/tmp/torch_extensions/test/main.cpp -o main.o
/tmp/torch_extensions/test/main.cpp: In function ‘int main()’:
/tmp/torch_extensions/test/main.cpp:2:23: error: expected ‘;’ before ‘}’ token
 int main() { return 0 }
                       ^
ninja: build stopped: subcommand failed.
```
which is a lot easier to read.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22445

Differential Revision: D16094205

Pulled By: ezyang

fbshipit-source-id: 21043344aac260dc3e4e04d6a42898507bb840e4
2019-07-09 16:41:42 -07:00
94bd5ddf7f Add some essentials for building c++ extensions on Windows (#22563)
Summary:
Fixes https://github.com/pytorch/pytorch/issues/22489.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22563

Differential Revision: D16142615

Pulled By: ezyang

fbshipit-source-id: d7c27a874f788dd27065fad6699485e4a6372ec4
2019-07-06 19:29:25 -07:00
693871ded3 Rename macros and build options NAMEDTENSOR_ENABLED to BUILD_NAMEDTENSOR (#22360)
Summary:
Currently the build system accepts USE_NAMEDTENSOR from the environment
variable and turns it into NAMEDTENSOR_ENABLED when passing to CMake.
This discrepancy does not seem necessary and complicates the build
system. The naming of this build option is also semantically incorrect
("BUILD_" vis-a-vis "USE_").  This commit eradicate this issue before it
is made into a stable release.

The support of NO_NAMEDTENSOR is also removed, since PyTorch has been
quite inconsistent about "NO_*" build options.

 ---

Note: All environment variables with their names starting with `BUILD_` are currently automatically passed to CMake with no need of an additional wrapper.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22360

Differential Revision: D16074509

Pulled By: zou3519

fbshipit-source-id: dc316287e26192118f3c99b945454bc50535b2ae
2019-07-02 11:46:13 -07:00
49481d576d Torch rename (#20774)
Summary:
This renames the CMake `caffe2` target to `torch`, as well as renaming `caffe2_gpu` to `torch_gpu` (and likewise for other gpu target variants).  Many intermediate variables that don't manifest as artifacts of the build remain for now with the "caffe2" name; a complete purge of `caffe2` from CMake variable names is beyond the scope of this PR.

The shell `libtorch` library that had been introduced as a stopgap in https://github.com/pytorch/pytorch/issues/17783 is again flattened in this PR.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/20774

Differential Revision: D15769965

Pulled By: kostmo

fbshipit-source-id: b86e8c410099f90be0468e30176207d3ad40c821
2019-06-12 20:12:34 -07:00
835a6b9da2 Fix namedtensor build (#21609)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/21609
ghimport-source-id: 648a0bcd28db2cdda1bf2fa6a904ca8f851088c2

Differential Revision: D15747687

Pulled By: zou3519

fbshipit-source-id: 2a972a15fa7399391617fc6e6b19879b86568c3a
2019-06-11 06:53:50 -07:00
f8aa6a8f44 Make a deep copy of extra_compile_flag dictionnary (#20221)
Summary:
See issue #20169
Pull Request resolved: https://github.com/pytorch/pytorch/pull/20221

Differential Revision: D15317126

Pulled By: ezyang

fbshipit-source-id: 0a12932db4f6ba15ea1d558fa329ce23fe2baef6
2019-05-13 08:11:39 -07:00
4ba28deb6e Unify libtorch and libcaffe2 (#17783)
Summary:
This PR is an intermediate step toward the ultimate goal of eliminating "caffe2" in favor of "torch".  This PR moves all of the files that had constituted "libtorch.so" into the "libcaffe2.so" library, and wraps "libcaffe2.so" with a shell library named "libtorch.so".  This means that, for now, `caffe2/CMakeLists.txt` becomes a lot bigger, and `torch/CMakeLists.txt` becomes smaller.

The torch Python bindings (`torch_python.so`) still remain in `torch/CMakeLists.txt`.

The follow-up to this PR will rename references to `caffe2` to `torch`, and flatten the shell into one library.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/17783

Differential Revision: D15284178

Pulled By: kostmo

fbshipit-source-id: a08387d735ae20652527ced4e69fd75b8ff88b05
2019-05-10 09:50:53 -07:00
3bfdffe487 Fix default CXX for Windows in cpp_extensions.py (#19052)
Summary:
Fixes https://github.com/pytorch/pytorch/issues/19017.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/19052

Differential Revision: D14846702

Pulled By: soumith

fbshipit-source-id: b0e4dadaa749da0fa2d0405a1a064820d094220a
2019-04-08 23:14:22 -07:00
e0c593eae7 detect C++ ABI flag for cpp extensions from available runtime information (#18994)
Summary:
Previously, when a user built PyTorch from source, but set the version string manually to be binary-formatted, it would've simply used CXX11_ABI=0 incorrectly.

We have this information available at runtime with `torch._C._GLIBCXX_USE_CXX11_ABI`, so this PR improves the situation by simply using that information.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18994

Differential Revision: D14839393

Pulled By: soumith

fbshipit-source-id: ca92e0810b29ffe688be82326e02a64a5649a3ad
2019-04-08 17:50:03 -07:00
d6d0fcc92b Add c10_cuda to libraries in CUDAExtension for Windows (#18982)
Summary:
This change was necessary for me to compile [apex](https://github.com/NVIDIA/apex) on Windows.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18982

Differential Revision: D14819818

Pulled By: soumith

fbshipit-source-id: 37ff9b93a72ab2b7c87f23a61e9f776c71c4c1a8
2019-04-06 10:30:51 -07:00
5ade96fc84 Update cpp_extension.py (#18638)
Summary:
Hi. It seems that when building CPP-extensions with CUDA for Windows, an `extra_cuda_cflags` options are not properly forwarded to `nvcc`.

Use of extra CUDA options is necessary to build, for instance, a InplaceABN (https://github.com/mapillary/inplace_abn), which requires `--expt-extended-lambda` option.

This PR adds one line that correctly appends `extra_cuda_cflags`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18638

Differential Revision: D14704270

Pulled By: ezyang

fbshipit-source-id: e1e330d193d9afd5707a5437a74c0499460d2b90
2019-04-02 07:56:38 -07:00
2b7a5d1876 don't include /usr/include when nvcc is in /usr/bin (#18127)
Summary:
...because gcc will have failures with very strange error messages
if you do.

This affects people with Debian/Ubuntu-provided NVCC, the PR should
not change anything for anyone else.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18127

Differential Revision: D14504386

Pulled By: soumith

fbshipit-source-id: 1aea168723cdc71cdcfffb3193ee116108ae755e
2019-03-18 12:18:27 -07:00
fe90ee9dc8 Add /MD to prevent linking errors on Windows
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/17799

Differential Revision: D14385777

Pulled By: ezyang

fbshipit-source-id: 8c1d9f80c48399087f5fae4474690e6d80d740e6
2019-03-08 10:46:25 -08:00
c78da0c6ed Enable using CMD when building cpp extensions on Windows
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/17706

Differential Revision: D14346482

Pulled By: ezyang

fbshipit-source-id: 7c85e51c701f6c0947ad324ef19fafda40ae1cb9
2019-03-06 14:45:31 -08:00
21193bf123 try to get rid of tmp_install (#16414)
Summary:
Rehash of previous attempts. This tries a different approach where we accept the install as specified in cmake (leaving bin/ include/ and lib/ alone), and then try to adjust the rest of the files to this more standard layout.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/16414

Differential Revision: D13863635

Pulled By: zdevito

fbshipit-source-id: 23725f5c64d7509bf3ca8f472dcdcad074de9828
2019-01-29 17:29:40 -08:00
c7ec7cdd46 Fixed syntax error in doctest (#15646)
Summary:
I fixed a very small extra parenthesis in a doctest.

I'm also going to use this issue as a place to propose the eventual inclusion of xdoctest (a pip installable library I wrote) in pytorch's test suite. I think there are a lot of problems with Python's built in doctest module, and I've built xdoctest to fix them. I would love for my project to get some exposure and its addition to PyTorch may benefit both projects. Please see the readme for more details on what xdoctest brings to the table over the builtin doctest module: https://github.com/Erotemic/xdoctest

I came across this small syntax error when working on ensuring xdoctest was compatible with pytorch. It isn't 100% there yet, but I'm working on it. My goal is to ensure that xdoctest is 100% compatible with all of torch's doctest out-of-the-box before writing up the PR. I'm also airing the idea out-loud before I commit too much time into this (or get my hopes up), so I'm attaching this little blurb to a no-brainer-merge PR to (1) demonstrate a little bit of value (because xdoctest flagged this syntax error) and (2) see how its received.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/15646

Differential Revision: D13606111

Pulled By: soumith

fbshipit-source-id: d4492801a38ee0ae64ea0326a83239cee4d811a4
2019-01-09 01:29:11 -08:00
0bf1383f0a Python <-> C++ Frontend inter-op (#13481)
Summary:
This PR enables C++ frontend modules to be bound into Python and added as submodules of Python modules. For this, I added lots of pybind11 bindings for the `torch::nn::Module` class, and modified the `torch.nn.Module` class in Python to have a new Metaclass that makes `isinstance(m, torch.nn.Module)` return true when `m` is a C++ frontend module. The methods and fields of C++ modules are bound in such a way that they work seamlessly as submodules of Python modules for most operations (one exception I know of: calling `.to()` ends up calling `.apply()` on each submodule with a Python lambda, which cannot be used in C++ -- this may require small changes on Python side).

I've added quite a bunch of tests to verify the bindings and equality with Python. I think I should also try out adding a C++ module as part of some large PyTorch module, like a WLM or something, and see if everything works smoothly.

The next step for inter-op across our system is ScriptModule <-> C++ Frontend Module inter-op. I think this will then also allow using C++ frontend modules from TorchScript.

apaszke zdevito

CC dzhulgakov
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13481

Differential Revision: D12981996

Pulled By: goldsborough

fbshipit-source-id: 147370d3596ebb0e94c82cec92993a148fee50a7
2018-12-13 08:04:02 -08:00
db15f2e13f Fix version.groups() (#14505)
Summary:
Fixes https://github.com/pytorch/pytorch/issues/14502

fmassa soumith
Pull Request resolved: https://github.com/pytorch/pytorch/pull/14505

Differential Revision: D13242386

Pulled By: goldsborough

fbshipit-source-id: faebae8795e1efd9c0ebc2294fe9648193d16624
2018-11-28 20:27:33 -08:00
6f2307ba6a Allow building libraries with setuptools that dont have abi suffix (#14130)
Summary:
When using `setuptools` to build a Python extension, setuptools will automatically add an ABI suffix like `cpython-37m-x86_64-linux-gnu` to the shared library name when using Python 3. This is required for extensions meant to be imported as Python modules. When we use setuptools to build shared libraries not meant as Python modules, for example libraries that define and register TorchScript custom ops, having your library called `my_ops.cpython-37m-x86_64-linux-gnu.so` is a bit annoying compared to just `my_ops.so`, especially since you have to reference the library name when loading it with `torch.ops.load_library` in Python.

This PR fixes this by adding a `with_options` class method to the `torch.utils.cpp_extension.BuildExtension` which allows configuring the `BuildExtension`. In this case, the first option we add is `no_python_abi_suffix`, which we then use in `get_ext_filename` (override from `setuptools.build_ext`) to throw away the ABI suffix.

I've added a test `setup.py` in a `no_python_abi_suffix_test` folder.

Fixes https://github.com/pytorch/pytorch/issues/14188

t-vi fmassa soumith
Pull Request resolved: https://github.com/pytorch/pytorch/pull/14130

Differential Revision: D13216575

Pulled By: goldsborough

fbshipit-source-id: 67dc345c1278a1a4ee4ca907d848bc1fb4956cfa
2018-11-27 17:35:53 -08:00
a13fd7ec28 Allow torch.utils.cpp_extension.load to load shared libraries that aren't Python modules (#13941)
Summary:
For custom TorchScript operators, `torch.ops.load_library` must be used and passed the path to the shared library containing the custom ops. Our C++ extensions stuff generally is meant to build a Python module and import it. This PR changes `torch.utils.cpp_extension.load` to have an option to just return the shared library path instead of importing it as a Python module, so you can then pass it to `torch.ops.load_library`. This means folks can re-use `torch.utils.cpp_extension.load` and `torch.utils.cpp_extension.load_inline` to even write their custom ops inline. I think t-vi  and fmassa will appreciate this.

soumith
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13941

Differential Revision: D13110592

Pulled By: goldsborough

fbshipit-source-id: 37756307dbf80a81d2ed550e67c8743dca01dc20
2018-11-26 09:39:21 -08:00
5b1b8682a3 Missing .decode() after check_output in cpp_extensions (#13935)
Summary:
soumith
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13935

Differential Revision: D13090852

Pulled By: goldsborough

fbshipit-source-id: 47da269d074fd1e7220e90580692d6ee489ec78b
2018-11-16 12:16:29 -08:00
2983998bb3 add torch-python target (#12742)
Summary:
This is the next minimal step towards moving _C into cmake. For now,
leave _C in setup.py, but reduce it to an empty stub file. All of its
sources are now part of the new torch-python cmake target.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12742

Reviewed By: soumith

Differential Revision: D13089691

Pulled By: anderspapitto

fbshipit-source-id: 1c746fda33cfebb26e02a7f0781fefa8b0d86385
2018-11-16 11:43:48 -08:00