* Fix public protobuf interface - wip
* Try turn on custom protobuf in mac jenkins.
* Adding back auto-fallback protobuf option
* Address typos pointed out by reviewers
* Remove OpenGL code from benchmark
* Make it possible to print plot in the ipython notbook
* Create the blob if the blob is not specified in the init net
* Do not use gf library for MKL. Even after I install the entire MKL library it is still not found. After removing it, the MKL code can still run
Summary:
After we removed android-cmake submodule and switched to android.cmake.toolchain from Android NDK, the code that builds cpufeatures dependency is no longer valid. This commit fixes it.
Closes https://github.com/caffe2/caffe2/pull/1957
Differential Revision: D6990082
Pulled By: Maratyszcza
fbshipit-source-id: ccbe8190e30e097474a2876ed4c0b263bcb117ef
Summary:
Previous behavior may fail to resolve the correct library name. A rework of https://github.com/caffe2/caffe2/pull/1935 as it was messed up in the rebase...
Closes https://github.com/caffe2/caffe2/pull/1950
Reviewed By: bddppq
Differential Revision: D6974530
Pulled By: yinghai
fbshipit-source-id: 924b653e8ac0b68c46341edfd3eb05d9cc0155f2
Summary:
To build with tests and benchmarks
`./scripts/build_android.sh -G Ninja -DBUILD_TEST=ON -DUSE_NNAPI=ON`
To run unit test
`adb push build_android/bin/nnapi_test data/local/tmp`
`adb shell "cd data/local/tmp &&./nnapi_test`
To run benchmark
`adb push build_android/bin/nnapi_benchmark data/local/tmp`
`adb shell "cd data/local/tmp &&./nnapi_benchmark`
Tested on Google PIxel 2 XL with android 8.1
Closes https://github.com/caffe2/caffe2/pull/1918
Reviewed By: Maratyszcza
Differential Revision: D6944604
Pulled By: hlu1
fbshipit-source-id: 462f010117ae4628b23bef506c41397de3817ad4
Summary:
Include six, enum34, and PeachPy as Caffe2 submodules, and use the versions from submodules instead of downloading them during configuration time
Closes https://github.com/caffe2/caffe2/pull/1917
Reviewed By: orionr
Differential Revision: D6938735
Pulled By: Maratyszcza
fbshipit-source-id: 841a6c47a1cd003a19f48f6c256aa4d9eb2cc6e4
Summary:
Original commit changeset: d0c1c7681605
Reverting due to broken OSS build due to this commit
Reviewed By: bddppq
Differential Revision: D6935666
fbshipit-source-id: 955cfeb6d5a4ed265b2e099094cfb5bfe960ff95
Summary:
Include six, enum34, and PeachPy as Caffe2 submodules, and use the versions from submodules instead of downloading them during configuration time
Closes https://github.com/caffe2/caffe2/pull/1901
Differential Revision: D6930731
Pulled By: Maratyszcza
fbshipit-source-id: d0c1c7681605d957de6f51bd24fbb25afc0f282f
Summary:
Addresses issue #1676
Now when `make install` is run, the `caffe2` (and `caffe`) python modules will be installed into the correct site-packages directory (relative to the prefix) instead of directly in the prefix.
Closes https://github.com/caffe2/caffe2/pull/1677
Reviewed By: pietern
Differential Revision: D6710247
Pulled By: bddppq
fbshipit-source-id: b49167d48fd94d87f7b7c1ebf0f187ec6a203470
Summary:
- Fix path to FXdiv and FP16 dependencies
- Link cpuinfo library
- Pull NNPACK fix for PYTHONPATH handling when launching PeachPy
- Pull cpuinfo fix for cross-compiling on Linux for Android
- Pull cpuinfo fix for CPUINFO_LIBRARY_TYPE support
- Pull cpuinfo fix for iOS builds
Closes https://github.com/caffe2/caffe2/pull/1869
Differential Revision: D6881428
Pulled By: Maratyszcza
fbshipit-source-id: 7b4115daa090096dbd97303503792e7b144fbb43
Summary:
On windows when using a prebuilt version of protobuf (such as provided by vcpkg) we need to set the PROTOBUF_LIBRARIES and PROTOBUF_INCLUDE_DIRS manually.
The CAFFE2_API decoration should only be defined to dllexport when building shared libs.
Closes https://github.com/caffe2/caffe2/pull/1854
Differential Revision: D6867345
Pulled By: Yangqing
fbshipit-source-id: d4d48f709d313af9dde103fc8dfbfc217261715b
Summary:
Last fix was uncommitted due to a bug in internal build (CAFFE2_API causing error). This one re-applies it as well as a few more, especially enabling gtest.
Earlier commit message: Basically, this should make windows {static_lib, shared_lib} * {static_runtime, shared_runtime} * {cpu, gpu} work other than gpu shared_lib, which willyd kindly pointed out a symbol limit problem. A few highlights:
(1) Updated newest protobuf.
(2) use protoc dllexport command to ensure proper symbol export for windows.
(3) various code updates to make sure that C2 symbols are properly shown
(4) cmake file changes to make build proper
(5) option to choose static runtime and shared runtime similar to protobuf
(6) revert to visual studio 2015 as current cuda and msvc 2017 do not play well together.
(7) enabled gtest and fixed testing bugs.
Earlier PR is #1793
Closes https://github.com/caffe2/caffe2/pull/1827
Differential Revision: D6832086
Pulled By: Yangqing
fbshipit-source-id: 85f86e9a992ee5c53c70b484b761c9d6aed721df
Summary:
This was removed in an earlier version. Anyway, I suspect this will make jenkins a bit unhappy (do we use gpu instances for building as well?) so firing a PR to test.
Closes https://github.com/caffe2/caffe2/pull/1833
Differential Revision: D6834889
Pulled By: Yangqing
fbshipit-source-id: bc501cdb9d83a32ad38d24e972c2bfec5242d767
Summary:
Historically, for interface dependent libraries (glog, gflags and protobuf), exposing them in Caffe2Config.cmake is usually difficult.
New versions of glog and gflags ship with new-style cmake targets, so one does not need to use variables. New-style targets also make it easier for people to depend on them in installed config files.
This diff modernizes the gflags library, and still provides a fallback path if the installed gflags does not have cmake config files coming with it.
It does change one behavior of the build process though - when one specifies -DUSE_GFLAGS=ON but gflags cannot be found, the old script automatically turns it off but the new script crashes, forcing the user to specify USE_GFLAGS=OFF.
Closes https://github.com/caffe2/caffe2/pull/1819
Differential Revision: D6826604
Pulled By: Yangqing
fbshipit-source-id: 210f3926f291c8bfeb24eb9671e5adfcbf8cf7fe
Summary:
This reverts commit d286264fccc72bf90a2fcd7da533ecca23ce557e
bypass-lint
An infra SEV is better than not reverting this diff.
If you copy this password, see you in SEV Review!
cause_a_sev_many_files
Differential Revision: D6817719
fbshipit-source-id: 8fe0ad7aba75caaa4c3cac5e0a804ab957a1b836
Summary:
Basically, this should make windows {static_lib, shared_lib} * {static_runtime, shared_runtime} * {cpu, gpu} work. A few highlights:
(1) Updated newest protobuf.
(2) use protoc dllexport command to ensure proper symbol export.
(3) various code updates to make sure that C2 symbols are properly shown
(4) cmake file changes to make build proper
(5) option to choose static runtime and shared runtime similar to protobuf
(6) revert to visual studio 2015 as current cuda and msvc 2017 do not play well together.
Closes https://github.com/caffe2/caffe2/pull/1793
Reviewed By: dzhulgakov
Differential Revision: D6817719
Pulled By: Yangqing
fbshipit-source-id: d286264fccc72bf90a2fcd7da533ecca23ce557e
Summary:
When system has protobuf package but hasn't protoc, cmake will be success:
> -- ******** Summary ********
-- General:
-- CMake version : 3.5.1
-- CMake command : /usr/bin/cmake
-- Git version : v0.8.1-967-g27d12d8-dirty
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 5.4.0
-- Protobuf compiler : PROTOBUF_PROTOC_EXECUTABLE-NOTFOUND
-- Protobuf include path : /usr/include
-- Protobuf libraries : optimized;/usr/lib/x86_64-linux-gnu/libprotobuf.so;debug;/usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread
...
Then make will be failed.
This submit make it to check protobuf package only when protoc has been found.
This pull request is a clone of [1781](https://github.com/caffe2/caffe2/pull/1781), that pull request closed by mistake.
Closes https://github.com/caffe2/caffe2/pull/1792
Differential Revision: D6800513
Pulled By: pietern
fbshipit-source-id: 79a77a139f342ae0aaa2c37fc1d9a74e28a08422
Summary:
Adds 2 features:
(1) In cmake, allow the use of -march=native
(2) During initialization, check if Caffe2 is built with matching cpu
features of the current machine.
This helps us guarding performance claims in case the Caffe2 baseline is
built with limited computation capability.
Currently only added avx, avx2 and fma which are common.
Closes https://github.com/caffe2/caffe2/pull/1775
Reviewed By: ezyang
Differential Revision: D6772059
Pulled By: Yangqing
fbshipit-source-id: 884a3d7c7a71ed9631b7c6269ae95d842a09e1bd
Summary:
This updates https://github.com/caffe2/caffe2/pull/1096/ to build doxygen docs with cmake and fixes operator catalog generation. See the new README.md for details, but you can run
```
mkdir build && cd build
cmake -DBUILD_DOCS=ON .. && make
```
and
```
python caffe2/python/docs/github.py ~/c2docs/_docs/operators-catalogue.md
```
to generate docs.
There was one weird issue in `generator.py` that we sometimes receive tuples and sometimes objects. I handled this just by testing `isinstance`, but we might want to be more principled in the future.
Closes https://github.com/caffe2/caffe2/pull/1758
Reviewed By: pietern
Differential Revision: D6752127
Pulled By: orionr
fbshipit-source-id: 9ba9ad8efc920b27a57327f8a7d3050f3650d4ce
Summary:
[x] Have to rebase
[x] Have to ensure this works on macOS + Anaconda
Closes https://github.com/caffe2/caffe2/pull/1741
Differential Revision: D6714172
Pulled By: pietern
fbshipit-source-id: 43a16d99a6ddf821a35b512c780cdfa35a721219
Summary:
This should fix Protobuf version problems on all Anaconda builds by putting include directories under Anaconda before all other include directories.
Closes https://github.com/caffe2/caffe2/pull/1728
Reviewed By: orionr
Differential Revision: D6698435
Pulled By: pjh5
fbshipit-source-id: f73f4a5ebb4ca91db14770a88a704ace69d37ba4
Summary:
This is in principle similar to #1612 and is tested on Windows 2017. CMake passes, although there are still bugs in the MSVC compiler that prevents cuda to compile properly.
The difference between this and #1612 is that this diff explicitly puts the CMake files into a separate folder and uses a MiscCheck.cmake chunk of code to test whether we need to include them. See README.txt for more details.
Closes https://github.com/caffe2/caffe2/pull/1727
Reviewed By: pietern
Differential Revision: D6693656
Pulled By: Yangqing
fbshipit-source-id: a74b0a1fde436d7bb2002a56affbc7bbb41ec621
Summary:
GCC version check is currently being skipped when using the
newly released CUDA 9.1.
This will also handle other CUDA 9.x minor releases if any,
reducing our work if there are such releases like 9.2. This
assumes that the next major CUDA version will be 10.0,
needing adjustment only after such major version is
released.
Closes https://github.com/caffe2/caffe2/pull/1658
Differential Revision: D6659000
Pulled By: pietern
fbshipit-source-id: 79291b5da9d4e8b4f2c7ac82fe2b1e7939438bc9
Summary:
`contrib/prof` provides functionality for profiling (eg. `prof_dag`) but no CMake.
Hence, provide CMake support for building it.
Reviewed By: Yangqing
Differential Revision: D6640488
fbshipit-source-id: 9ed8095b10d7c0337db061206daf2a66f41f4713
Summary:
Make operator QuantDecompZstd buildable in open source. The operator is not built by default. Need to specify -DBUILD_SHARE_DIR=ON -DUSE_ZSTD=ON to build it.
Test plans: Build android caffe2 with the change without issue. Run a model with the operator successfully.
Closes https://github.com/caffe2/caffe2/pull/1613
Reviewed By: Yangqing
Differential Revision: D6556723
Pulled By: sf-wind
fbshipit-source-id: 453a7d787a55928f2dea1ed2b99f2df011aa8d26
Summary:
Thanks to feldim2425 we know that GCC 5 in Ubuntu 17.04 and later
doesn't define the macro _GLIBCXX_USE_C99 and by extension the
std::to_string, std::stoi, and std::stod functions (and probably
more). Instead of avoiding using these functions, we simply recommend
people to use GCC 6 or higher on the newer Ubuntu versions where GCC 5
doesn't work.
As a side note, CUDA 8.0 is compatible with GCC up to version 5. This
implies that compiling Caffe2 with CUDA on Ubuntu >= 17.10 implies
using CUDA >= 9.0. If you need to compile with CUDA 8.0 and are on
Ubuntu, you are stuck on version 16.04 or lower.
I verified this fix by running cmake on Ubuntu 17.10 with
-DCMAKE_CXX_COMPILER=/usr/bin/g++5 and observing the fatal error.
This closes#1633.
Closes https://github.com/caffe2/caffe2/pull/1645
Differential Revision: D6620812
Pulled By: pietern
fbshipit-source-id: 29af88cad9bede4fd952084c404c85db05baa9c4
Summary:
This means warnings and errors fire sooner rather than later.
This requires a fix for an issue where CMAKE_REQUIRED_FLAGS propagates
to some unrelated check, which then fails, because the Android
compiler doesn't support -mavx2.
Closes https://github.com/caffe2/caffe2/pull/1646
Differential Revision: D6620129
Pulled By: pietern
fbshipit-source-id: 4d1185406ebee3a523d39811bca6783bee82c898
Summary:
This fixes the in-tree protoc build on CentOS 7 (that ships with super old protobuf version).
Closes https://github.com/caffe2/caffe2/pull/1595
Differential Revision: D6529307
Pulled By: pietern
fbshipit-source-id: ac81c7cd884846854b4ffd4909377e87d93bddc3
Summary:
our cmake build used to link against libpython.so with its absolute path (instead of -LSOME_LIB_PATH -lpython), so at runtime loader will think it needs the libpython.so at that specific path, and so load in an additional libpython.so, which causes the python binding built with one python installation not reusable by another (maybe on same machine or sometimes even not on same machine). The solution is quite simple, which is we don't link against libpython, leave all the python related symbols unresolved at build time, they will be resolved at runtime when imported into python.
Closes https://github.com/caffe2/caffe2/pull/1514
Reviewed By: dzhulgakov
Differential Revision: D6412405
Pulled By: bddppq
fbshipit-source-id: 9ff5b752ae3806bfac94085942f82d89c304c887
Summary:
The plural version is not defined in the CentOS CMake module.
Verified EIGEN3_INCLUDE_DIR is defined in the Ubuntu CMake module.
This fixes the build on CentOS when using system Eigen3.
Closes https://github.com/caffe2/caffe2/pull/1505
Differential Revision: D6390712
Pulled By: pietern
fbshipit-source-id: b8abb14a62e0ff9fa9c920866504da0e75786c0d
Summary:
This should also be ported to Gloo since its Cuda.cmake was
synchronized to Caffe2 in #1256.
Verified that running CMake with `-DCUDA_ARCH_NAME=Manual` and
`-DCUDA_ARCH_BIN=70` ends up running nvcc with `-gencode
arch=compute_70,code=sm_70`.
Closes#1460.
Closes https://github.com/caffe2/caffe2/pull/1487
Reviewed By: bwasti
Differential Revision: D6376222
Pulled By: pietern
fbshipit-source-id: 563a2947567a2af8a0e64475b346a19d76545ed3
Summary:
Also bumped third_party/protobuf to v3.4.1 similar to #1462 . cc pietern
Closes https://github.com/caffe2/caffe2/pull/1466
Reviewed By: pietern
Differential Revision: D6322210
Pulled By: Yangqing
fbshipit-source-id: 00f72472b71d1903a2705daf56652e4fb3fc021e
Summary:
This is in order for us to share compression ops to oss.
Closes https://github.com/caffe2/caffe2/pull/1463
Reviewed By: hlu1
Differential Revision: D6319101
Pulled By: Yangqing
fbshipit-source-id: 16c94e71fc3efe256054a648170aaf7702e5bcfe
Summary:
This correctly adds handling of CUDA 8.0 and 9.0 by cmake.
**Discussion:**
CUDA 9.0 is currently not handled by cmake. When trying to build
with it and gcc6, the following cmake error is shown:
-- CUDA detected: 9.0
...
CMake Error at cmake/Dependencies.cmake:332 (message):
CUDA 8.0 is not compatible with GCC version >= 6. Use the following option
to use another version (for example):
-DCUDA_HOST_COMPILER=/usr/bin/gcc-5
Closes https://github.com/caffe2/caffe2/pull/1392
Differential Revision: D6317033
Pulled By: pietern
fbshipit-source-id: 08b89f21b994af52533d5afaaa62f26e2e94aee8
Summary:
Do not try to link against `libcblas.so` when using the OpenBLAS
back-end. This fixes#763.
I briefly checked the OpenBLAS repository and as far as I can tell, the OpenBLAS build script by build never created a library called _cblas_.
Closes https://github.com/caffe2/caffe2/pull/1420
Differential Revision: D6283019
Pulled By: pietern
fbshipit-source-id: 53cd4455bdc63ee9f31d5bca9822844548350ae3
Summary:
Few people complained in NNPACK repo about broken build on PPC64, as it specifically whitelists supported architecture in its CMakeLists.txt, and refuses to build on unsupported platforms. This commit explicitly disables NNPACK build (as part of Caffe2 build) on unsupported architectures.
Closes https://github.com/caffe2/caffe2/pull/1439
Differential Revision: D6288999
Pulled By: Maratyszcza
fbshipit-source-id: 76c40e9ce882356944b63968df8fd853f21ecd35
