frozenleaves/pytorch
1
0
Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2025-10-20 12:54:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
26740f853e10e34a7e8d2b72b950875796350418
pytorch/c10/util
History
mingfeima a427d90411 add int4 packed gemm support on CPU device (#117475) 
This patch adds int4 packed gemm support on CPU, both `avx512` and `avx2` are supported. It is used to speedup https://github.com/pytorch-labs/gpt-fast

The default perf measured on Intel (R) Xeon (R) CPU Max 9480, single socket (56 cores) is `16.13 sec total, 12.40 tokens/sec`

* WOQ int4 on avx512: `5.92 sec total, 33.79 tokens/sec`
* WOQ int4 on avx2: `6.90 sec total, 29.00 tokens/sec`

WOQ int4 is measured with method: https://github.com/pytorch-labs/gpt-fast?tab=readme-ov-file#int4-weight-only-quantization

Pull Request resolved: https://github.com/pytorch/pytorch/pull/117475
Approved by: https://github.com/jgong5, https://github.com/malfet
2024-03-06 16:25:53 +00:00
..
AbortHandler.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
accumulate.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
AlignOf.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
ApproximateClock.cpp
[c10] Move profiler clock to libc10 for timestamps (#111972)
2023-10-27 16:18:40 +00:00
ApproximateClock.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
Array.h
Move array_of helper to c10/util (#116749)
2024-01-04 21:58:32 +00:00
ArrayRef.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
Backtrace.cpp
[Fix]: remove unnecessary copies in aten, c10, and torch bindings (#90629)
2022-12-12 17:05:52 +00:00
Backtrace.h
Revert "[1/N] Cleanup header inclusions in torch_cpu by iwyu (#101178)"
2023-09-25 20:05:25 +00:00
BFloat16-inl.h
migrate PyTorch to c10::bit_cast (#98418)
2023-04-06 19:38:06 +00:00
BFloat16-math.h
Remove use of math_compat.h (#116167)
2024-01-19 03:37:55 +00:00
BFloat16.h
Add missing cstdint includes (#116458)
2023-12-29 18:30:26 +00:00
bit_cast.h
[5/N] Fixes clang-tidy warnings in c10/{core,util}/*.h (#115354)
2023-12-09 17:16:04 +00:00
bits.h
Add various uninterpreted bit tensor data types (try 2) (#95860)
2023-03-04 03:35:59 +00:00
Bitset.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
BUILD.bazel
extract out //c10/util:base library (#70854)
2022-01-26 11:51:45 +00:00
build.bzl
migrate PyTorch to c10::bit_cast (#98418)
2023-04-06 19:38:06 +00:00
C++17.cpp
…
C++17.h
[ROCm] replace ROCmLoops.cuh with hipified CUDALoops.cuh (#120101)
2024-02-22 21:57:36 +00:00
CallOnce.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
complex_math.cpp
Remove use of math_compat.h (#116167)
2024-01-19 03:37:55 +00:00
complex_math.h
[codemod][lowrisk] Remove extra semi colon from caffe2/c10/util/Float8_e5m2.h (#115761)
2024-01-04 02:02:26 +00:00
complex_utils.h
Logcumsumexp for CPU (#93153)
2023-01-27 22:29:33 +00:00
complex.h
[C10] Mark Complex::imag as C10_HOST_DEVICE (#116877)
2024-01-05 21:17:05 +00:00
ConstexprCrc.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
copysign.h
Remove use of math_compat.h (#116167)
2024-01-19 03:37:55 +00:00
DeadlockDetection.cpp
Enable nested namespace check in clang-tidy (#118506)
2024-01-31 00:32:35 +00:00
DeadlockDetection.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
Deprecated.h
[PyTorch] Autoformat c10 (#56830)
2021-04-30 21:23:28 -07:00
DimVector.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
env.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
Exception.cpp
[Reland 1] Cleanup header inclusions in torch_cpu by iwyu (#112311)
2023-11-19 04:06:36 +00:00
Exception.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
ExclusivelyOwned.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
ExclusivelyOwnedTensorTraits.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
FbcodeMaps.h
[PyTorch] add c10/util/FbcodeMaps.h (#96359)
2023-03-10 02:18:16 +00:00
flags_use_gflags.cpp
adjust header inclusions in C10 as sugguested by IWYU (#102467)
2023-05-31 19:19:10 +00:00
flags_use_no_gflags.cpp
More tidy fixes (#93069)
2023-01-27 06:40:50 +00:00
Flags.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
flat_hash_map.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
Float8_e4m3fn-inl.h
[2/N] Fixes clang-tidy warnings in header files (#113727)
2023-11-16 13:21:15 +00:00
Float8_e4m3fn.cpp
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
Float8_e4m3fn.h
Remove unused c10/util/C++17.h inclusion and outdated checks (#120149)
2024-02-17 14:28:17 +00:00
Float8_e4m3fnuz-inl.h
additional support for float8_e4m3fnuz and _e5m2fnuz (#115214)
2024-01-22 18:33:41 +00:00
Float8_e4m3fnuz.cpp
additional support for float8_e4m3fnuz and _e5m2fnuz (#115214)
2024-01-22 18:33:41 +00:00
Float8_e4m3fnuz.h
Remove unused c10/util/C++17.h inclusion and outdated checks (#120149)
2024-02-17 14:28:17 +00:00
Float8_e5m2-inl.h
[2/N] Fixes clang-tidy warnings in header files (#113727)
2023-11-16 13:21:15 +00:00
Float8_e5m2.cpp
Add torch.float8_e5m2 and torch.float8_e4m3 data types (#104242)
2023-07-20 16:09:11 +00:00
Float8_e5m2.h
[codemod][lowrisk] Remove extra semi colon from caffe2/c10/util/Float8_e5m2.h (#115761)
2024-01-04 02:02:26 +00:00
Float8_e5m2fnuz-inl.h
additional support for float8_e4m3fnuz and _e5m2fnuz (#115214)
2024-01-22 18:33:41 +00:00
Float8_e5m2fnuz.cpp
additional support for float8_e4m3fnuz and _e5m2fnuz (#115214)
2024-01-22 18:33:41 +00:00
Float8_e5m2fnuz.h
Remove unused c10/util/C++17.h inclusion and outdated checks (#120149)
2024-02-17 14:28:17 +00:00
Float8_fnuz_cvt.h
additional support for float8_e4m3fnuz and _e5m2fnuz (#115214)
2024-01-22 18:33:41 +00:00
floating_point_utils.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
FunctionRef.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
generic_math.h
Fix c10::div_floor_floating compile error (#115647)
2023-12-20 00:09:01 +00:00
Half-inl.h
Construct c10::Half from float16_t on ARMv8 (#120425)
2024-02-23 04:22:45 +00:00
Half.cpp
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
Half.h
Construct c10::Half from float16_t on ARMv8 (#120425)
2024-02-23 04:22:45 +00:00
hash.h
improve CUDACachingAllocator lock contention (#118550)
2024-02-04 16:45:25 +00:00
IdWrapper.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
int128.cpp
increase clang-tidy coverage to more c10 source files (#102902)
2023-06-04 06:33:01 +00:00
int128.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
intrusive_ptr.cpp
…
intrusive_ptr.h
c10:intrusive_ptr, self assignment (#119275)
2024-03-06 01:11:56 +00:00
irange.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
LeftRight.cpp
…
LeftRight.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
llvmMathExtras.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
Load.h
[2/N] Fixes clang-tidy warnings in header files (#113727)
2023-11-16 13:21:15 +00:00
logging_is_google_glog.h
Fix TORCH_CHECK macros when glog is used (#83216)
2022-08-11 15:27:45 +00:00
logging_is_not_google_glog.h
[RELAND] Remove some unnecessary <iostream> includes from headers (#108150)
2023-09-20 21:55:15 +00:00
Logging.cpp
Enable nested namespace check in clang-tidy (#118506)
2024-01-31 00:32:35 +00:00
Logging.h
[BE] Delete GCC-7 ICE workarounds (#120122)
2024-02-20 00:31:20 +00:00
MathConstants.cpp
Use c10 math constants consistently in Math.h (#91967)
2023-01-12 11:53:52 +00:00
MathConstants.h
Use c10 math constants consistently in Math.h (#91967)
2023-01-12 11:53:52 +00:00
MaybeOwned.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
Metaprogramming.cpp
…
Metaprogramming.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
numa.cpp
Enable misc clang-tidy checks (#110283)
2023-09-30 10:39:52 +00:00
numa.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
Optional.cpp
[PyTorch] Redirect c10::optional to std::optional (#101995)
2023-11-30 02:46:41 +00:00
Optional.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
OptionalArrayRef.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
order_preserving_flat_hash_map.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
overloaded.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
ParallelGuard.cpp
Avoid COW materialization in at::parallel_for/parallel_reduce (#120455)
2024-03-01 05:05:28 +00:00
ParallelGuard.h
Avoid COW materialization in at::parallel_for/parallel_reduce (#120455)
2024-03-01 05:05:28 +00:00
python_stub.h
Move pyobj_ to TensorImpl (#18225)
2019-03-23 12:50:38 -07:00
qint8.h
[pytorch] add missing host-device attribute to fix clang build (#37358)
2020-04-27 18:24:20 -07:00
qint32.h
[pytorch] add missing host-device attribute to fix clang build (#37358)
2020-04-27 18:24:20 -07:00
quint2x4.h
[oss][pytorch] Add quint2x4 dtype (#65545)
2021-10-06 14:22:00 -07:00
quint4x2.h
[PyTorch] Autoformat c10 (#56830)
2021-04-30 21:23:28 -07:00
quint8.h
Fix misleading doc string in quint8.h (#48418)
2020-11-25 20:48:39 -08:00
Registry.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
safe_numerics.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
ScopeExit.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
signal_handler.cpp
use condition_variable and wait_until in nccl dump on timeout (#120544)
2024-02-28 00:06:08 +00:00
signal_handler.h
use condition_variable and wait_until in nccl dump on timeout (#120544)
2024-02-28 00:06:08 +00:00
SmallBuffer.h
[Clang-tidy header][21/N] Fix clang-tidy warnings in aten/src/ATEN/*.{cpp,h} (#120763)
2024-03-03 23:18:43 +00:00
SmallVector.cpp
increase clang-tidy coverage to more c10 source files (#102902)
2023-06-04 06:33:01 +00:00
SmallVector.h
[7/N] Fixes clang-tidy warnings in c10/{core,util}/*.h (#115495)
2023-12-19 02:14:30 +00:00
sparse_bitset.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
ssize.h
[10/N] Fixes clang-tidy warnings in c10/util/*.h (#116326)
2023-12-23 04:59:55 +00:00
static_tracepoint_elfx86.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
static_tracepoint.h
[caffe2][SDT] Check whether TORCH_DISABLE_SDT macro is defined before referencing it (#107149)
2023-08-16 19:52:12 +00:00
strides.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
string_utils.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
string_view.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
StringUtil.cpp
[Caffe2] Fix bug in str on wide types (#117531)
2024-01-29 20:11:37 +00:00
StringUtil.h
[Caffe2] Fix bug in str on wide types (#117531)
2024-01-29 20:11:37 +00:00
strong_type.h
Consistent compute numel/contiguous strategy with SymInts (#85858)
2022-09-30 21:26:34 +00:00
Synchronized.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
tempfile.cpp
[Reland] fix c10:TempFile APIs on Windows (#108508)
2023-09-10 16:58:41 +00:00
tempfile.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
thread_name.cpp
c10/util/thread_name.cpp: pthread_setname_np requires Glibc 2.12 (#55063)
2021-05-21 10:26:51 -07:00
thread_name.h
Canonicalize includes in c10, and add tests for it (#36299)
2020-04-10 12:07:52 -07:00
ThreadLocal.h
Include Macros.h in ThreadLocal
2021-05-27 20:47:56 -07:00
ThreadLocalDebugInfo.cpp
cleanup unused include (#93359)
2023-02-04 02:15:50 +00:00
ThreadLocalDebugInfo.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
Type_demangle.cpp
[Reland 1] Cleanup header inclusions in torch_cpu by iwyu (#112311)
2023-11-19 04:06:36 +00:00
Type_no_demangle.cpp
[2/N] Cleanup header inclusions in torch_cpu by iwyu (#109964)
2023-11-19 20:56:32 +00:00
Type.h
[T153220354] Fix header inclusions in c10 (#1541) (#101846)
2023-05-20 19:35:14 +00:00
TypeCast.cpp
Avoid inlining error reporting in checked_convert (#66721)
2021-10-20 16:04:09 -07:00
TypeCast.h
[Reland] [11/N] Enable clang-tidy warnings on c10/util/*.h (#116751)
2024-01-08 11:07:58 +00:00
typeid.cpp
[PyTorch] Further reduce cost of TypeMeta::_typeMetaData (by 10x!) (#98105)
2023-04-12 17:44:48 +00:00
typeid.h
Add unsigned integer dtypes to PyTorch (#116594)
2024-01-07 07:40:49 +00:00
TypeIndex.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
TypeList.cpp
…
TypeList.h
Remove unused c10/util/C++17.h inclusion and outdated checks (#120149)
2024-02-17 14:28:17 +00:00
TypeSafeSignMath.h
Revert "[11/N] Enable clang-tidy warnings on c10/util/*.h (#116353)"
2024-01-03 22:22:11 +00:00
TypeTraits.cpp
…
TypeTraits.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
Unicode.cpp
[PyTorch] Autoformat c10 (#56830)
2021-04-30 21:23:28 -07:00
Unicode.h
[PyTorch] Autoformat c10 (#56830)
2021-04-30 21:23:28 -07:00
UniqueVoidPtr.cpp
Enable nested namespace check in clang-tidy (#118506)
2024-01-31 00:32:35 +00:00
UniqueVoidPtr.h
[9/N] Fixes clang-tidy warnings in c10/util/*.h (#116185)
2023-12-22 09:35:44 +00:00
Unroll.h
add int4 packed gemm support on CPU device (#117475)
2024-03-06 16:25:53 +00:00
win32-headers.h
[PyTorch] Autoformat c10 (#56830)
2021-04-30 21:23:28 -07:00