Summary:
This PR changes the cmake of Caffe2 to look for system dependencies before resorting to the submodules in `third-party`. Only googletest should logically be in third-party, the other libraries should ideally be installed as system dependencies by the user. This PR adds system dependency checks for Gloo, CUB, pybind11, Eigen and benchmark, as these were missing from the cmake files.
In addition it removes the execution of `git submodule update --init` in cmake. This seems like bad behavior to me, it should be up to the user to download submodules and manage the git repository.
Closes https://github.com/caffe2/caffe2/pull/382
Differential Revision: D5124123
Pulled By: Yangqing
fbshipit-source-id: cc34dda58ffec447874a89d01058721c02a52476
Summary:
After the change we will be able to simply define targets and find dependencies.
Closes https://github.com/caffe2/caffe2/pull/640
Differential Revision: D5121700
Pulled By: Yangqing
fbshipit-source-id: 2d21e1afbccb09614054feccdd1bef55cbe3b035
Summary: Adding a simple video data layer which allows to read video data from frames, videos and output 5D tensor. It also allows multiple labels. The current implementation is based on ffmpeg
Differential Revision: D4801798
fbshipit-source-id: 46448e9c65fb055c2d71855447383a33ade0e444
Summary:
Otherwise compilation fails pretty far into the build, which is inconvenient.
The error reported when trying to compile with GCC 6:
CUDA 8.0 is not compatible with GCC version >= 6. Use the following
options to configure GCC version 5:
-DCMAKE_CXX_COMPILER=/usr/bin/g++-5
-DCMAKE_C_COMPILER=/usr/bin/gcc-5
-DCUDA_HOST_COMPILER:FILEPATH=/usr/bin/gcc-5
Closes https://github.com/caffe2/caffe2/pull/504
Reviewed By: akyrola
Differential Revision: D5004299
Pulled By: pietern
fbshipit-source-id: 185cd2f846f291a48e1d41ce0d87ca69e7f2c593
Summary:
This PR is based on commit "977c6b3" as this version allows MKL to use all the cores available.
All MKL related files are added here after incorporating review comments, major changes include
1. usage of Clang-format(Linter) with --style = Google
2. usage of macros for checking input and filter dimension in the mkl operators
3. merged Max and Average pooling functions
4. created a new folder for mkl related python scripts in Python folder and moved them there
5. there is no mkl_alexnet_test.py as that was redundant while convnet_benchmark.py does the same thing
Closes https://github.com/caffe2/caffe2/pull/270
Differential Revision: D4905219
Pulled By: Yangqing
fbshipit-source-id: e5f5b189714a835b93b9ebda24c52e09572dfca7
Summary:
Cannot guarantee Gloo will build on 32-bit systems as we don't run continuous build/test for this.
Verified this works by changing 8 to 7 and observing USE_GLOO defaulting to OFF.
Closes https://github.com/caffe2/caffe2/pull/401
Differential Revision: D4943135
Pulled By: pietern
fbshipit-source-id: 1972658afe819951e24ffbec76eb615c36ab0cc2
Summary:
When installing on systems such as Arch Linux where the default python version is 3 the build will fail. To fix this instead of changing the python link in the shell it is more efficient to set the default python version allowed by cmake.
Closes https://github.com/caffe2/caffe2/pull/361
Differential Revision: D4932214
Pulled By: Yangqing
fbshipit-source-id: 06997d2df68b8e4037d72fd49813f6f74ca7591b
Summary:
This tiny patch fix missing ```CUDA_NVCC_FLAGS``` & ```CUDA_HOST_ARCH``` from ```caffe_detect_installed_gpus()```.
-----------------
People may want define their custom flags or compilers that are more CUDA compatible. Automatic gpu arch detection ignores these flags and fail. Example of such custom flags:
```
cmake . \
-DCUDA_ARCH_NAME="Auto" \
-DCUDA_HOST_COMPILER="/usr/bin/gcc5"
```
* Autodetection part fails regardless proper compiler flags are passed, due to system gcc 7.0 that doesnt work with CUDA thus all arch will be enabled:
```
-- The C compiler identification is GNU 7.0.1
-- The CXX compiler identification is GNU 7.0.1
...//\\...
-- CUDA detected: 8.0
...//\\...
-- Automatic GPU detection failed. Building for all known architectures.
-- Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50 sm_60 sm_61
```
* Patch fix the autodetection time as expected:
```
$ cmake ../ -DCUDA_NVCC_FLAGS="-Xcompiler=-std=c++03 -I/usr/include/cuda/"
-- The C compiler identification is
Closes https://github.com/caffe2/caffe2/pull/288
Differential Revision: D4914215
Pulled By: Yangqing
fbshipit-source-id: c407a750e03cb163f9d57f9f6403042704046014
Summary:
This was found necessary on some CentOS. aaronmarkham
Closes https://github.com/caffe2/caffe2/pull/240
Differential Revision: D4819591
Pulled By: Yangqing
fbshipit-source-id: 40161cd484a2c8d43f26077919ad2762440dde13
Summary:
This also requires a change to cmake/External/nccl.cmake to use the
static NCCL binary instead of the shared object. When the Caffe2/Gloo
build uses the bundled NCCL version it should be packaged up in the
resulting libraries and not cause another runtime dependency on a
library that has to be installed separately.
Closes https://github.com/caffe2/caffe2/pull/218
Differential Revision: D4769926
Pulled By: pietern
fbshipit-source-id: 5c85559992c200d874f4218724823815ffb5adb5
Summary:
aaronmarkham this solves your Windows build issue. Basically:
(1) VS 2017 does not have CUDA support yet, and we will be waiting on NVidia to do so.
(2) VS 2015 and 2017 need different cmake generator strings.
This PR shows how to determine those and also updates appveyor to do contbuild guard for the following 3 settings:
- VS2015 without cuda
- VS2017 without cuda
- VS2015 with cuda
Closes https://github.com/caffe2/caffe2/pull/210
Differential Revision: D4745007
Pulled By: Yangqing
fbshipit-source-id: 50952552843abd0eb6f4145d9f132daeee3a6794
Summary:
Necessary if CXX isn't set when cmake is called. The CXX variable will then be
empty which prevents make from using its own default.
Closes https://github.com/caffe2/caffe2/pull/202
Differential Revision: D4711113
Pulled By: Yangqing
fbshipit-source-id: 895c07044b263ba9b5440453978248506d7ac225
Summary:
(Note: previous revert was due to a race condition between D4657831 and
D4659953 that I failed to catch.)
After this, we should have contbuild guarding the Windows build both with
and without CUDA.
This includes a series of changes that are needed to make Windows build,
specifically:
(1) Various flags that are needed in the cmake system, specially dealing
with /MD, /MT, cuda, cudnn, whole static linking, etc.
(2) Contbuild scripts based on appveyo.
(3) For Windows build, note that one will need to use "cmake --build" to
build stuff so that the build type is consistent between configuration and
actual build. see scripts\build_windows.bat for details.
(4) In logging.h, ERROR is already defined by Windows. I don't have a good
solution now, and as a result, LOG(ERROR) on windows is going to be
LOG(INFO).
(5) variable length array is not supported by MSVC (and it is not part of
C++ standard). As a result I replaced them with vectors.
(6) sched.h is not available on Windows, so akyrola 's awesome simple
async net might encounter some slowdown due to no affinity setting on
Windows.
(7) MSVC has a bug that does not work very well with template calls inide
a templated function call, which is a known issue that should be fixed in
MSVC 2017. However for now this means changes to conv_op_impl.h and
recurrent_net_op.h. No actual functionalities are changed.
(8) std host function calls are not supported in CUDA8+MSVC, so I changed
lp_pool (and maybe a few others) to use cuda device functions.
(9) The current Scale and Axpy has heavy templating that does not work
well with MSVC. As a result I reverted azzolini 's changes to the Scale
and Axpy interface, moved the fixed-length version to ScaleFixedSize and
AxpyFixedSize.
(10) CUDA + MSVC does not deal with Eigen well, so I guarded all Eigen
parts to only the non-CUDA part.
(11) In conclusion, it is fun but painful to deal with visual c++.
Differential Revision: D4666745
fbshipit-source-id: 3c9035083067bdb19a16d9c345c1ce66b6a86600
Summary:
After this, we should have contbuild guarding the Windows build both with
and without CUDA.
This includes a series of changes that are needed to make Windows build,
specifically:
(1) Various flags that are needed in the cmake system, specially dealing
with /MD, /MT, cuda, cudnn, whole static linking, etc.
(2) Contbuild scripts based on appveyo.
(3) For Windows build, note that one will need to use "cmake --build" to
build stuff so that the build type is consistent between configuration and
actual build. see scripts\build_windows.bat for details.
(4) In logging.h, ERROR is already defined by Windows. I don't have a good
solution now, and as a result, LOG(ERROR) on windows is going to be
LOG(INFO).
(5) variable length array is not supported by MSVC (and it is not part of
C++ standard). As a result I replaced them with vectors.
(6) sched.h is not available on Windows, so akyrola 's awesome simple
async net might encounter some slowdown due to no affinity setting on
Windows.
(7) MSVC has a
Closes https://github.com/caffe2/caffe2/pull/183
Reviewed By: ajtulloch
Differential Revision: D4657831
Pulled By: Yangqing
fbshipit-source-id: 070ded372ed78a7e3e3919fdffa1d337640f146e
Summary:
MSVC 2015 has known bugs about template functions so these changes aim to fix them - no functional differences introduced.
Closes https://github.com/caffe2/caffe2/pull/179
Reviewed By: ajtulloch
Differential Revision: D4635241
Pulled By: Yangqing
fbshipit-source-id: a282a96e1e626e9440c1e3f3cb15b5b1fa710887
Summary:
(1) Since cub seems to be a better memory pool I made cnmem optional.
(2) Added MKL testing since Intel now provides an apt source, but that doesn't seem to work right now.
(3) Added cmake file for nervana gpu.
Closes https://github.com/caffe2/caffe2/pull/175
Differential Revision: D4627056
Pulled By: Yangqing
fbshipit-source-id: 9676fa32fce2a29574c0bf7e9d31660b5535cb51
Summary:
This clears up a bunch of windows build errors, but there are still 12 errors mostly relating to
- template keywords
- initializer list
- pthreadpool
that are not readily available on windows. Also, cuda build is being disabled right now.
Current error can be found here: https://ci.appveyor.com/project/Yangqing/caffe2-w2ucm
Closes https://github.com/caffe2/caffe2/pull/151
Reviewed By: bwasti
Differential Revision: D4564591
Pulled By: Yangqing
fbshipit-source-id: adacad5fa2d6d52d586700947972e3674e3b6e60
Summary:
This is essentially https://github.com/caffe2/caffe2/pull/146/ but shipit
failed to trigger task determinator.
Reviewed By: bwasti
Differential Revision: D4557698
fbshipit-source-id: b0e6777957e76df4e23671371098c2c6fe83b55c
Summary: There are still a lot to clean up, but this is a start change.
Reviewed By: bwasti
Differential Revision: D4543980
fbshipit-source-id: 757fc49db230b56996f02d5de9b69030ebbf3b77
Summary:
Turns out that building on raspbian is easy as a cake for caffe2 - cmake is awesome.
Closes https://github.com/caffe2/caffe2/pull/112
Differential Revision: D4480985
Pulled By: Yangqing
fbshipit-source-id: 5dbe5e1e71d8680dea7a5ec8a9ce7fbe6aa5270a
Summary: This fixes build that include caffe2 and change the value of CMAKE_BINARY_DIR to their own binary dir. Allows the generation of protobuf headers/files in particular.
Reviewed By: Yangqing
Differential Revision: D4466126
fbshipit-source-id: eba264094dd2bff07a7f050b95fd2d5525462b09
Summary:
This will help issues like #99
Closes https://github.com/caffe2/caffe2/pull/101
Differential Revision: D4448397
Pulled By: Yangqing
fbshipit-source-id: ede3fafc1b1314886583e8ea38948bb31e69347b
Summary: Github import didn't work and the manual import lost some files.
Reviewed By: Yangqing
Differential Revision: D4408509
fbshipit-source-id: ec8edb8c02876410f0ef212bde6847a7ba327fe4
