From 9c957723a0fedd9c637e63e023a613019e2cab60 Mon Sep 17 00:00:00 2001 From: Klaus Zimmermann Date: Tue, 2 Sep 2025 15:49:19 +0200 Subject: [PATCH] Replace setup.py develop with pip install -e (#156710) #156027 already replaced most use of `python setup.py develop`. This PR only adds a few more occurrences. Pull Request resolved: https://github.com/pytorch/pytorch/pull/156710 Approved by: https://github.com/atalman --- .ci/pytorch/macos-test.sh | 4 ++-- CONTRIBUTING.md | 24 ++++++++++++------------ cmake/Dependencies.cmake | 6 +++--- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/.ci/pytorch/macos-test.sh b/.ci/pytorch/macos-test.sh index c56066e6b596..79d47da43171 100755 --- a/.ci/pytorch/macos-test.sh +++ b/.ci/pytorch/macos-test.sh @@ -199,7 +199,7 @@ torchbench_setup_macos() { git checkout "$(cat ../.github/ci_commit_pins/vision.txt)" git submodule update --init --recursive python setup.py clean - python setup.py develop + python -m pip install -e . -v --no-build-isolation popd pushd torchaudio @@ -208,7 +208,7 @@ torchbench_setup_macos() { git submodule update --init --recursive python setup.py clean #TODO: Remove me, when figure out how to make TorchAudio find brew installed openmp - USE_OPENMP=0 python setup.py develop + USE_OPENMP=0 python -m pip install -e . -v --no-build-isolation popd checkout_install_torchbench diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dade8f4ec6ec..632fb4712dc0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -88,13 +88,13 @@ source venv/bin/activate # or `& .\venv\Scripts\Activate.ps1` on Windows * If you want to have no-op incremental rebuilds (which are fast), see [Make no-op build fast](#make-no-op-build-fast) below. -* When installing with `python -m pip install -e .` (in contrast to `python -m pip install .`) Python runtime will use +* When installing with `python -m pip install -e . -v --no-build-isolation` (in contrast to `python -m pip install . -v --no-build-isolation`) Python runtime will use the current local source-tree when importing `torch` package. (This is done by creating [`.egg-link`](https://wiki.python.org/moin/PythonPackagingTerminology#egg-link) file in `site-packages` folder) This way you do not need to repeatedly install after modifying Python files (`.py`). However, you would need to reinstall if you modify Python interface (`.pyi`, `.pyi.in`) or non-Python files (`.cpp`, `.cc`, `.cu`, `.h`, ...). - One way to avoid running `python -m pip install -e .` every time one makes a change to C++/CUDA/ObjectiveC files on Linux/Mac, + One way to avoid running `python -m pip install -e . -v --no-build-isolation` every time one makes a change to C++/CUDA/ObjectiveC files on Linux/Mac, is to create a symbolic link from `build` folder to `torch/lib`, for example, by issuing following: ```bash pushd torch/lib; sh -c "ln -sf ../../build/lib/libtorch_cpu.* ."; popd @@ -116,7 +116,7 @@ source venv/bin/activate # or `& .\venv\Scripts\Activate.ps1` on Windows Next run `python setup.py clean`. After that, you can install in editable mode again. -* If you run into errors when running `python -m pip install -e .`, here are some debugging steps: +* If you run into errors when running `python -m pip install -e . -v --no-build-isolation`, here are some debugging steps: 1. Run `printf '#include \nint main() { printf("Hello World");}'|clang -x c -; ./a.out` to make sure your CMake works and can compile this simple Hello World program without errors. 2. Nuke your `build` directory. The `setup.py` script compiles binaries into the `build` folder and caches many @@ -132,7 +132,7 @@ source venv/bin/activate # or `& .\venv\Scripts\Activate.ps1` on Windows python -m pip install -r requirements.txt python -m pip install --no-build-isolation -v -e . ``` - 4. The main step within `python -m pip install -e .` is running `cmake --build build` from the `build` directory. If you want to + 4. The main step within `python -m pip install -e . -v --no-build-isolation` is running `make` from the `build` directory. If you want to experiment with some environment variables, you can pass them into the command: ```bash ENV_KEY1=ENV_VAL1[, ENV_KEY2=ENV_VAL2]* CMAKE_FRESH=1 python -m pip install --no-build-isolation -v -e . @@ -645,9 +645,9 @@ can be selected interactively with your mouse to zoom in on a particular part of the program execution timeline. The `--native` command-line option tells `py-spy` to record stack frame entries for PyTorch C++ code. To get line numbers for C++ code it may be necessary to compile PyTorch in debug mode by prepending -your `python -m pip install -e .` call to compile PyTorch with `DEBUG=1`. -Depending on your operating system it may also be necessary to run `py-spy` with -root privileges. +your `python -m pip install -e . -v --no-build-isolation` call to compile +PyTorch with `DEBUG=1`. Depending on your operating system it may also be +necessary to run `py-spy` with root privileges. `py-spy` can also work in an `htop`-like "live profiling" mode and can be tweaked to adjust the stack sampling rate, see the `py-spy` readme for more @@ -655,10 +655,10 @@ details. ## Managing multiple build trees -One downside to using `python -m pip install -e .` is that your development -version of PyTorch will be installed globally on your account (e.g., if -you run `import torch` anywhere else, the development version will be -used). +One downside to using `python -m pip install -e . -v --no-build-isolation` is +that your development version of PyTorch will be installed globally on your +account (e.g., if you run `import torch` anywhere else, the development version +will be used). If you want to manage multiple builds of PyTorch, you can make use of [venv environments](https://docs.python.org/3/library/venv.html) to maintain @@ -719,7 +719,7 @@ options. ### Code completion and IDE support -When using `python -m pip install -e .`, PyTorch will generate +When using `python -m pip install -e . -v --no-build-isolation`, PyTorch will generate a `compile_commands.json` file that can be used by many editors to provide command completion and error highlighting for PyTorch's C++ code. You need to `pip install ninja` to generate accurate diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cmake index 3354c18dd3af..e4e82b16f410 100644 --- a/cmake/Dependencies.cmake +++ b/cmake/Dependencies.cmake @@ -1666,9 +1666,9 @@ if(USE_KINETO) set(CMAKE_REQUIRED_LINK_OPTIONS "") if(NOT EXCEPTIONS_WORK) message(FATAL_ERROR - "Detected that statically linking against CUPTI causes exceptions to stop working. " - "See https://github.com/pytorch/pytorch/issues/57744 for more details. " - "Perhaps try: USE_CUPTI_SO=1 CMAKE_FRESH=1 python setup.py develop") + "Detected that statically linking against CUPTI causes exceptions to stop working. " + "See https://github.com/pytorch/pytorch/issues/57744 for more details. " + "Perhaps try: USE_CUPTI_SO=1 CMAKE_FRESH=1 python -m pip install -e . -v --no-build-isolation") endif() endif()