frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 21:14:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

third_party: Fix build_bundled script

Browse Source 
This script was hanging up on a couple of licenses not being parsed
correctly / not accounted for in the original pass through

Should resolve issues with tests not running 100% correctly

Signed-off-by: Eli Uriegas <eliuriegasfb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76336

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Approved by: https://github.com/ezyang
This commit is contained in:
Eli Uriegas
2022-04-25 13:27:25 -07:00
committed by PyTorch MergeBot
parent da984c507c
commit a9a17ce626
3 changed files with 184 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitattributes vendored
View File

@ -3,3 +3,4 @@
.github/workflows/generated-*.yml linguist-generated=true .github/workflows/generated-*.yml linguist-generated=true
.github/generated-* linguist-generated=true .github/generated-* linguist-generated=true
.github/scripts/gql_mocks.json linguist-generated=true .github/scripts/gql_mocks.json linguist-generated=true
third_party/LICENSES_BUNDLED.txt linguist-generated=true

189
third_party/LICENSES_BUNDLED.txt generated vendored
View File

@ -6,11 +6,21 @@ License: MIT
Files: third_party/FP16 Files: third_party/FP16
For details, see third_party/FP16/LICENSE For details, see third_party/FP16/LICENSE
Name: FP16-source
License: MIT
Files: third_party/XNNPACK/build/FP16-source
For details, see third_party/XNNPACK/build/FP16-source/LICENSE
Name: FXdiv Name: FXdiv
License: MIT License: MIT
Files: third_party/FXdiv Files: third_party/FXdiv
For details, see third_party/FXdiv/LICENSE For details, see third_party/FXdiv/LICENSE
Name: FXdiv-source
License: MIT
Files: third_party/XNNPACK/build/FXdiv-source
For details, see third_party/XNNPACK/build/FXdiv-source/LICENSE
Name: NNPACK Name: NNPACK
License: BSD-2-Clause License: BSD-2-Clause
Files: third_party/NNPACK Files: third_party/NNPACK
@ -29,22 +39,36 @@ Files: third_party/XNNPACK
Name: benchmark Name: benchmark
License: Apache-2.0 License: Apache-2.0
Files: third_party/benchmark, Files: third_party/benchmark,
third_party/protobuf/third_party/benchmark, third_party/onnx/third_party/benchmark,
third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark, third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark,
third_party/onnx/third_party/benchmark third_party/protobuf/third_party/benchmark
For details, see third_party/benchmark/LICENSE, For details, see third_party/benchmark/LICENSE,
third_party/protobuf/third_party/benchmark/LICENSE, third_party/onnx/third_party/benchmark/LICENSE,
third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark/LICENSE, third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark/LICENSE,
third_party/onnx/third_party/benchmark/LICENSE third_party/protobuf/third_party/benchmark/LICENSE
Name: breakpad
License: BSD-3-Clause
Files: third_party/breakpad
For details, see third_party/breakpad/LICENSE
Name: clog Name: clog
License: BSD-2-Clause License: BSD-2-Clause
Files: third_party/cpuinfo/deps/clog, Files: third_party/QNNPACK/deps/clog,
third_party/fbgemm/third_party/cpuinfo/deps/clog, third_party/XNNPACK/build/clog-source/deps/clog,
third_party/QNNPACK/deps/clog third_party/XNNPACK/build/cpuinfo-source/deps/clog,
For details, see third_party/cpuinfo/deps/clog/LICENSE, third_party/cpuinfo/deps/clog,
third_party/fbgemm/third_party/cpuinfo/deps/clog/LICENSE, third_party/fbgemm/third_party/cpuinfo/deps/clog
third_party/QNNPACK/deps/clog/LICENSE For details, see third_party/QNNPACK/deps/clog/LICENSE,
third_party/XNNPACK/build/clog-source/deps/clog/LICENSE,
third_party/XNNPACK/build/cpuinfo-source/deps/clog/LICENSE,
third_party/cpuinfo/deps/clog/LICENSE,
third_party/fbgemm/third_party/cpuinfo/deps/clog/LICENSE
Name: clog-source
License: BSD-2-Clause
Files: third_party/XNNPACK/build/clog-source
For details, see third_party/XNNPACK/build/clog-source/LICENSE
Name: cpuinfo Name: cpuinfo
License: BSD-2-Clause License: BSD-2-Clause
@ -53,6 +77,21 @@ Files: third_party/cpuinfo,
For details, see third_party/cpuinfo/LICENSE, For details, see third_party/cpuinfo/LICENSE,
third_party/fbgemm/third_party/cpuinfo/LICENSE third_party/fbgemm/third_party/cpuinfo/LICENSE
Name: cpuinfo-source
License: BSD-2-Clause
Files: third_party/XNNPACK/build/cpuinfo-source
For details, see third_party/XNNPACK/build/cpuinfo-source/LICENSE
Name: cudnn_frontend
License: MIT
Files: third_party/cudnn_frontend
For details, see third_party/cudnn_frontend/LICENSE.txt
Name: dart
License: Apache-2.0
Files: third_party/flatbuffers/dart
For details, see third_party/flatbuffers/dart/LICENSE
Name: eigen Name: eigen
License: BSD-3-Clause License: BSD-3-Clause
Files: third_party/eigen Files: third_party/eigen
@ -68,12 +107,17 @@ License: BSD-3-Clause
Files: third_party/fbgemm Files: third_party/fbgemm
For details, see third_party/fbgemm/LICENSE For details, see third_party/fbgemm/LICENSE
Name: flatbuffers
License: Apache-2.0
Files: third_party/flatbuffers
For details, see third_party/flatbuffers/LICENSE.txt
Name: fmt Name: fmt
License: MIT with exception License: MIT with exception
Files: third_party/kineto/libkineto/third_party/fmt, Files: third_party/fmt,
third_party/fmt third_party/kineto/libkineto/third_party/fmt
For details, see third_party/kineto/libkineto/third_party/fmt/LICENSE.rst, For details, see third_party/fmt/LICENSE.rst,
third_party/fmt/LICENSE.rst third_party/kineto/libkineto/third_party/fmt/LICENSE.rst
Name: foxi Name: foxi
License: MIT License: MIT
@ -87,14 +131,18 @@ Files: third_party/gemmlowp/gemmlowp
Name: generator Name: generator
License: Apache-2.0 License: Apache-2.0
Files: third_party/kineto/libkineto/third_party/googletest/googlemock/scripts/generator, Files: third_party/XNNPACK/build/googletest-source/googlemock/scripts/generator,
third_party/googletest/googlemock/scripts/generator, third_party/benchmark/build/third_party/googletest/src/googlemock/scripts/generator,
third_party/fbgemm/third_party/googletest/googlemock/scripts/generator, third_party/fbgemm/third_party/googletest/googlemock/scripts/generator,
third_party/googletest/googlemock/scripts/generator,
third_party/kineto/libkineto/third_party/googletest/googlemock/scripts/generator,
third_party/protobuf/third_party/googletest/googlemock/scripts/generator, third_party/protobuf/third_party/googletest/googlemock/scripts/generator,
third_party/tensorpipe/third_party/googletest/googlemock/scripts/generator third_party/tensorpipe/third_party/googletest/googlemock/scripts/generator
For details, see third_party/kineto/libkineto/third_party/googletest/googlemock/scripts/generator/LICENSE, For details, see third_party/XNNPACK/build/googletest-source/googlemock/scripts/generator/LICENSE,
third_party/googletest/googlemock/scripts/generator/LICENSE, third_party/benchmark/build/third_party/googletest/src/googlemock/scripts/generator/LICENSE,
third_party/fbgemm/third_party/googletest/googlemock/scripts/generator/LICENSE, third_party/fbgemm/third_party/googletest/googlemock/scripts/generator/LICENSE,
third_party/googletest/googlemock/scripts/generator/LICENSE,
third_party/kineto/libkineto/third_party/googletest/googlemock/scripts/generator/LICENSE,
third_party/protobuf/third_party/googletest/googlemock/scripts/generator/LICENSE, third_party/protobuf/third_party/googletest/googlemock/scripts/generator/LICENSE,
third_party/tensorpipe/third_party/googletest/googlemock/scripts/generator/LICENSE third_party/tensorpipe/third_party/googletest/googlemock/scripts/generator/LICENSE
@ -103,46 +151,58 @@ License: BSD-3-Clause
Files: third_party/gloo Files: third_party/gloo
For details, see third_party/gloo/LICENSE For details, see third_party/gloo/LICENSE
Name: googlebenchmark-source
License: Apache-2.0
Files: third_party/XNNPACK/build/googlebenchmark-source
For details, see third_party/XNNPACK/build/googlebenchmark-source/LICENSE
Name: googlemock Name: googlemock
License: BSD-3-Clause License: BSD-3-Clause
Files: third_party/kineto/libkineto/third_party/googletest/googlemock, Files: third_party/XNNPACK/build/googletest-source/googlemock,
third_party/googletest/googlemock,
third_party/fbgemm/third_party/googletest/googlemock, third_party/fbgemm/third_party/googletest/googlemock,
third_party/kineto/libkineto/third_party/googletest/googlemock,
third_party/protobuf/third_party/googletest/googlemock, third_party/protobuf/third_party/googletest/googlemock,
third_party/tensorpipe/third_party/googletest/googlemock third_party/tensorpipe/third_party/googletest/googlemock
For details, see third_party/kineto/libkineto/third_party/googletest/googlemock/LICENSE, For details, see third_party/XNNPACK/build/googletest-source/googlemock/LICENSE,
third_party/googletest/googlemock/LICENSE,
third_party/fbgemm/third_party/googletest/googlemock/LICENSE, third_party/fbgemm/third_party/googletest/googlemock/LICENSE,
third_party/kineto/libkineto/third_party/googletest/googlemock/LICENSE,
third_party/protobuf/third_party/googletest/googlemock/LICENSE, third_party/protobuf/third_party/googletest/googlemock/LICENSE,
third_party/tensorpipe/third_party/googletest/googlemock/LICENSE third_party/tensorpipe/third_party/googletest/googlemock/LICENSE
Name: googletest Name: googletest
License: BSD-3-Clause License: BSD-3-Clause
Files: third_party/kineto/libkineto/third_party/googletest, Files: third_party/XNNPACK/build/googletest-source/googletest,
third_party/kineto/libkineto/third_party/googletest/googletest,
third_party/googletest,
third_party/googletest/googletest,
third_party/fbgemm/third_party/googletest, third_party/fbgemm/third_party/googletest,
third_party/fbgemm/third_party/googletest/googletest, third_party/fbgemm/third_party/googletest/googletest,
third_party/googletest,
third_party/kineto/libkineto/third_party/googletest,
third_party/kineto/libkineto/third_party/googletest/googletest,
third_party/protobuf/third_party/googletest, third_party/protobuf/third_party/googletest,
third_party/protobuf/third_party/googletest/googletest, third_party/protobuf/third_party/googletest/googletest,
third_party/tensorpipe/third_party/googletest, third_party/tensorpipe/third_party/googletest,
third_party/tensorpipe/third_party/googletest/googletest third_party/tensorpipe/third_party/googletest/googletest
For details, see third_party/kineto/libkineto/third_party/googletest/LICENSE, For details, see third_party/XNNPACK/build/googletest-source/googletest/LICENSE,
third_party/kineto/libkineto/third_party/googletest/googletest/LICENSE,
third_party/googletest/LICENSE,
third_party/googletest/googletest/LICENSE,
third_party/fbgemm/third_party/googletest/LICENSE, third_party/fbgemm/third_party/googletest/LICENSE,
third_party/fbgemm/third_party/googletest/googletest/LICENSE, third_party/fbgemm/third_party/googletest/googletest/LICENSE,
third_party/googletest/LICENSE,
third_party/kineto/libkineto/third_party/googletest/LICENSE,
third_party/kineto/libkineto/third_party/googletest/googletest/LICENSE,
third_party/protobuf/third_party/googletest/LICENSE, third_party/protobuf/third_party/googletest/LICENSE,
third_party/protobuf/third_party/googletest/googletest/LICENSE, third_party/protobuf/third_party/googletest/googletest/LICENSE,
third_party/tensorpipe/third_party/googletest/LICENSE, third_party/tensorpipe/third_party/googletest/LICENSE,
third_party/tensorpipe/third_party/googletest/googletest/LICENSE third_party/tensorpipe/third_party/googletest/googletest/LICENSE
Name: googletest-source
License: BSD-3-Clause
Files: third_party/XNNPACK/build/googletest-source
For details, see third_party/XNNPACK/build/googletest-source/LICENSE
Name: gtest Name: gtest
License: BSD-3-Clause License: BSD-3-Clause
Files: third_party/ideep/mkl-dnn/tests/gtests/gtest Files: third_party/ideep/mkl-dnn/tests/gtest,
For details, see third_party/ideep/mkl-dnn/tests/gtests/gtest/LICENSE third_party/ideep/mkl-dnn/third_party/oneDNN/tests/gtests/gtest
For details, see third_party/ideep/mkl-dnn/tests/gtest/LICENSE,
third_party/ideep/mkl-dnn/third_party/oneDNN/tests/gtests/gtest/LICENSE
Name: ideep Name: ideep
License: MIT License: MIT
@ -154,11 +214,21 @@ License: BSD-3-Clause
Files: third_party/ios-cmake Files: third_party/ios-cmake
For details, see third_party/ios-cmake/LICENSE For details, see third_party/ios-cmake/LICENSE
Name: json
License: MIT
Files: third_party/cudnn_frontend/include/contrib/nlohmann/json
For details, see third_party/cudnn_frontend/include/contrib/nlohmann/json/LICENSE.txt
Name: kineto Name: kineto
License: BSD-3-Clause License: BSD-3-Clause
Files: third_party/kineto Files: third_party/kineto
For details, see third_party/kineto/LICENSE For details, see third_party/kineto/LICENSE
Name: libdisasm
License: Clarified Artistic License
Files: third_party/breakpad/src/third_party/libdisasm
For details, see third_party/breakpad/src/third_party/libdisasm/LICENSE
Name: libnop Name: libnop
License: Apache-2.0 License: Apache-2.0
Files: third_party/tensorpipe/third_party/libnop Files: third_party/tensorpipe/third_party/libnop
@ -169,6 +239,11 @@ License: MIT
Files: third_party/tensorpipe/third_party/libuv Files: third_party/tensorpipe/third_party/libuv
For details, see third_party/tensorpipe/third_party/libuv/LICENSE For details, see third_party/tensorpipe/third_party/libuv/LICENSE
Name: lss
License: BSD-3-Clause
Files: third_party/breakpad/src/third_party/lss
For details, see third_party/breakpad/src/third_party/lss/LICENSE
Name: miniz-2.0.8 Name: miniz-2.0.8
License: MIT License: MIT
Files: third_party/miniz-2.0.8 Files: third_party/miniz-2.0.8
@ -189,12 +264,20 @@ License: BSD-Source-Code
Files: third_party/neon2sse Files: third_party/neon2sse
For details, see third_party/neon2sse/LICENSE For details, see third_party/neon2sse/LICENSE
Name: oneDNN
License: Apache-2.0
Files: third_party/ideep/mkl-dnn/third_party/oneDNN
For details, see third_party/ideep/mkl-dnn/third_party/oneDNN/LICENSE
Name: onnx
License: Apache-2.0
Files: third_party/onnx
For details, see third_party/onnx/LICENSE
Name: onnx Name: onnx
License: MIT License: MIT
Files: third_party/onnx-tensorrt/third_party/onnx, Files: third_party/onnx-tensorrt/third_party/onnx
third_party/onnx For details, see third_party/onnx-tensorrt/third_party/onnx/LICENSE
For details, see third_party/onnx-tensorrt/third_party/onnx/LICENSE,
third_party/onnx/LICENSE
Name: onnx-tensorrt Name: onnx-tensorrt
License: MIT License: MIT
@ -208,23 +291,30 @@ Files: third_party/protobuf
Name: psimd Name: psimd
License: MIT License: MIT
Files: third_party/psimd Files: third_party/XNNPACK/deps/psimd,
For details, see third_party/psimd/LICENSE third_party/psimd
For details, see third_party/XNNPACK/deps/psimd/LICENSE,
third_party/psimd/LICENSE
Name: pthreadpool Name: pthreadpool
License: BSD-2-Clause License: BSD-2-Clause
Files: third_party/pthreadpool Files: third_party/pthreadpool
For details, see third_party/pthreadpool/LICENSE For details, see third_party/pthreadpool/LICENSE
Name: pthreadpool-source
License: BSD-2-Clause
Files: third_party/XNNPACK/build/pthreadpool-source
For details, see third_party/XNNPACK/build/pthreadpool-source/LICENSE
Name: pybind11 Name: pybind11
License: BSD-3-Clause License: BSD-3-Clause
Files: third_party/pybind11, Files: third_party/onnx/third_party/pybind11,
third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11, third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11,
third_party/onnx/third_party/pybind11, third_party/pybind11,
third_party/tensorpipe/third_party/pybind11 third_party/tensorpipe/third_party/pybind11
For details, see third_party/pybind11/LICENSE, For details, see third_party/onnx/third_party/pybind11/LICENSE,
third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/LICENSE, third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/LICENSE,
third_party/onnx/third_party/pybind11/LICENSE, third_party/pybind11/LICENSE,
third_party/tensorpipe/third_party/pybind11/LICENSE third_party/tensorpipe/third_party/pybind11/LICENSE
Name: python-peachpy Name: python-peachpy
@ -242,6 +332,21 @@ License: BSL-1.0
Files: third_party/sleef Files: third_party/sleef
For details, see third_party/sleef/LICENSE.txt For details, see third_party/sleef/LICENSE.txt
Name: src
License: BSD-3-Clause
Files: third_party/benchmark/build/third_party/googletest/src
For details, see third_party/benchmark/build/third_party/googletest/src/LICENSE
Name: swift
License: Apache-2.0
Files: third_party/flatbuffers/swift
For details, see third_party/flatbuffers/swift/LICENSE
Name: tb_plugin
License: BSD-3-Clause
Files: third_party/kineto/tb_plugin
For details, see third_party/kineto/tb_plugin/LICENSE
Name: tbb Name: tbb
License: Apache-2.0 License: Apache-2.0
Files: third_party/tbb Files: third_party/tbb

54
third_party/build_bundled.py vendored
View File

@ -1,4 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import argparse
import os import os
@ -52,7 +53,7 @@ def create_bundled(d, outstream):
outstream.write(f"Files: {files}\n") outstream.write(f"Files: {files}\n")
outstream.write(' For details, see ') outstream.write(' For details, see ')
outstream.write(license_file) outstream.write(license_file)
outstream.write('\n\n') outstream.write('\n\n')
def identify_license(f, exception=''): def identify_license(f, exception=''):
@ -89,6 +90,8 @@ def identify_license(f, exception=''):
elif 'BoostSoftwareLicense-Version1.0' in txt: elif 'BoostSoftwareLicense-Version1.0' in txt:
# Hmm, do we need to check the text? # Hmm, do we need to check the text?
return 'BSL-1.0' return 'BSL-1.0'
elif squeeze("Clarified Artistic License") in txt:
return 'Clarified Artistic License'
elif all([squeeze(m) in txt.lower() for m in bsd3_txt]): elif all([squeeze(m) in txt.lower() for m in bsd3_txt]):
return 'BSD-3-Clause' return 'BSD-3-Clause'
elif all([squeeze(m) in txt.lower() for m in bsd3_v1_txt]): elif all([squeeze(m) in txt.lower() for m in bsd3_v1_txt]):
@ -97,30 +100,30 @@ def identify_license(f, exception=''):
return 'BSD-2-Clause' return 'BSD-2-Clause'
elif all([squeeze(m) in txt.lower() for m in bsd3_src_txt]): elif all([squeeze(m) in txt.lower() for m in bsd3_src_txt]):
return 'BSD-Source-Code' return 'BSD-Source-Code'
elif all([squeeze(m) in txt.lower() for m in mit_txt]): elif any([squeeze(m) in txt.lower() for m in mit_txt]):
return 'MIT' return 'MIT'
else: else:
raise ValueError('unknown license') raise ValueError('unknown license')
mit_txt = ['permission is hereby granted, free of charge, to any person ' mit_txt = ['permission is hereby granted, free of charge, to any person ',
'obtaining a copy of this software and associated documentation ' 'obtaining a copy of this software and associated documentation ',
'files (the "software"), to deal in the software without ' 'files (the "software"), to deal in the software without ',
'restriction, including without limitation the rights to use, copy, ' 'restriction, including without limitation the rights to use, copy, ',
'modify, merge, publish, distribute, sublicense, and/or sell copies ' 'modify, merge, publish, distribute, sublicense, and/or sell copies ',
'of the software, and to permit persons to whom the software is ' 'of the software, and to permit persons to whom the software is ',
'furnished to do so, subject to the following conditions:', 'furnished to do so, subject to the following conditions:',
'the above copyright notice and this permission notice shall be ' 'the above copyright notice and this permission notice shall be ',
'included in all copies or substantial portions of the software.', 'included in all copies or substantial portions of the software.',
'the software is provided "as is", without warranty of any kind, ' 'the software is provided "as is", without warranty of any kind, ',
'express or implied, including but not limited to the warranties of ' 'express or implied, including but not limited to the warranties of ',
'merchantability, fitness for a particular purpose and ' 'merchantability, fitness for a particular purpose and ',
'noninfringement. in no event shall the authors or copyright holders ' 'noninfringement. in no event shall the authors or copyright holders ',
'be liable for any claim, damages or other liability, whether in an ' 'be liable for any claim, damages or other liability, whether in an ',
'action of contract, tort or otherwise, arising from, out of or in ' 'action of contract, tort or otherwise, arising from, out of or in ',
'connection with the software or the use or other dealings in the ' 'connection with the software or the use or other dealings in the ',
'software.' 'software.',
] ]
bsd3_txt = ['redistribution and use in source and binary forms, with or without ' bsd3_txt = ['redistribution and use in source and binary forms, with or without '
@ -154,6 +157,21 @@ bsd3_src_txt = bsd3_txt[:2] + bsd3_txt[4:]
if __name__ == '__main__': if __name__ == '__main__':
third_party = os.path.join(mydir) third_party = os.path.join(mydir)
fname = os.path.join(third_party, 'LICENSES_BUNDLED.txt') parser = argparse.ArgumentParser(
description="Generate bundled licenses file",
)
parser.add_argument(
"--out-file",
type=str,
default=os.environ.get(
"PYTORCH_THIRD_PARTY_BUNDLED_LICENSE_FILE",
str(os.path.join(third_party, 'LICENSES_BUNDLED.txt'))
),
help="location to output new bundled licenses file",
)
args = parser.parse_args()
fname = args.out_file
print(f"+ Writing bundled licenses to {args.out_file}")
with open(fname, 'w') as fid: with open(fname, 'w') as fid:
create_bundled(third_party, fid) create_bundled(third_party, fid)