Commit Graph

22 Commits

Author SHA1 Message Date
13fbf21a76 [nativert] Port string join and split to c10/util (#152873)
Summary:
Torch Native Runtime RFC: https://github.com/pytorch/rfcs/pull/72
Port string utils functions join and split to c10/util

Test Plan:
Added tests in `string_util_test.cpp`
buck2 run mode/opt caffe2/c10/test:util_base_tests

Differential Revision: D74202473

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152873
Approved by: https://github.com/cyyever, https://github.com/Skylion007
2025-05-07 03:58:11 +00:00
5a66c1d921 [nativert] Add utility function to convert strings into numbers. (#151467)
Summary:

nativert RFC: https://github.com/zhxchen17/rfcs/blob/master/RFC-0043-torch-native-runtime.md

To land the runtime into PyTorch core, we will gradually land logical parts of the code into the Github issue and get each piece properly reviewed.

This diff adds a small library to convert strings into numbers which will later be used for parsing graph IR.

Differential Revision: D73133034

## Test Plan

c10 unittests

Pull Request resolved: https://github.com/pytorch/pytorch/pull/151467
Approved by: https://github.com/cyyever, https://github.com/albanD
2025-04-30 21:20:52 +00:00
82ce888273 c10::string_view -> std::string_view in more places (#142517)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/142517
Approved by: https://github.com/malfet
2024-12-12 19:45:59 +00:00
fca0f34b83 Switch c10::string_view to std::string_view (#139635)
Shortens `string_view_starts_with` to `starts_with`. Adds some missing headers. Isolates `c10_string_view` to use with `get_fully_qualified_name`.

Test Plan: Sandcastle

Reviewed By: ezyang

Differential Revision: D64833558

Pull Request resolved: https://github.com/pytorch/pytorch/pull/139635
Approved by: https://github.com/Skylion007, https://github.com/ezyang
2024-11-27 01:41:18 +00:00
cyy
3907f36808 Turn some variables and functions into static (#136847)
Re-check some files and mark variables and functions into static and fix other warnings.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/136847
Approved by: https://github.com/ezyang
2024-10-29 17:01:56 +00:00
cyy
f5f1d0a753 Fix build warnings for torch_python (#134981)
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/134981
Approved by: https://github.com/ezyang
2024-09-12 03:59:34 +00:00
7ef7c265d4 Ack codecvt_utf8_utf16 as a deprecated func in C++17 (#127659)
https://en.cppreference.com/w/cpp/header/codecvt.  This starts to fail on MacOS after migrating it to MacOS 14 with a newer toolchain.  For example 57baae9c9b.

As there is no clear alternative to the deprecated function yet, I just ack the warning to fix the build and complete the migration https://github.com/pytorch/pytorch/issues/127490
Pull Request resolved: https://github.com/pytorch/pytorch/pull/127659
Approved by: https://github.com/kit1980, https://github.com/atalman
2024-06-01 04:31:39 +00:00
e0d04b7119 [Caffe2] Fix bug in str on wide types (#117531)
Summary:
The current implementation of `str` passes wide types (`wchar_t`, `wchar_t*`, `std::wstring`) directly to `std::ostringstream`. This has the following behavior:

 - C++17, `wchar_t` & `wchar_t *`: print the integer representation of the character or the pointer. This is unexpected and almost certainly a (runtime) bug.
 - C++17, `std::wstring`: compile-time error.
 - C++20, all of the above: compile-time error.

To fix the bug and to enable C++20 migration, this diff performs narrowing on these wide types (assuming UTF-16 encoding) before passing them to `std::ostringstream`. This fixes both the C++20 compile time errors and the C++17 runtime bugs.

This bug surfaced in enabling C++20 windows builds, because windows specific caffe2 code uses `TORCH_CHECK` with wide strings, which references `str` for generating error messages.

Test Plan: CI & https://godbolt.org/z/ecTGd8Ma9

Differential Revision: D52792393

Pull Request resolved: https://github.com/pytorch/pytorch/pull/117531
Approved by: https://github.com/malfet
2024-01-29 20:11:37 +00:00
cyy
bae61ecb96 [Reland 1] Cleanup header inclusions in torch_cpu by iwyu (#112311)
Reland https://github.com/pytorch/pytorch/pull/101178 to use IWYU on torch_cpu. The header file changes are excluded to avoid breaking internal jobs.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/112311
Approved by: https://github.com/ezyang
2023-11-19 04:06:36 +00:00
83deaa16ed Revert "[1/N] Cleanup header inclusions in torch_cpu by iwyu (#101178)"
This reverts commit b7a95f4fdb8a79dc459cc757dafcdbd0953b1a62.

Reverted https://github.com/pytorch/pytorch/pull/101178 on behalf of https://github.com/atalman due to Break internal CI ([comment](https://github.com/pytorch/pytorch/pull/101178#issuecomment-1734384645))
2023-09-25 20:05:25 +00:00
cyy
b7a95f4fdb [1/N] Cleanup header inclusions in torch_cpu by iwyu (#101178)
Following our previous IWYU work  #100304 on C10, it makes more sense to try IWYU on torch_cpu. This PR does exactly that. Meanwhile, it fixes issue #48684.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/101178
Approved by: https://github.com/ezyang
2023-09-24 05:01:20 +00:00
c3bcf5f628 Support multiple separator characters, '/' and '\\', on Windows. (#98146)
On Windows, both '/' and '\\' can be used as a path separator, so `StripBasename` should handle them as path separators.

`StripBasename` is used in the `is_enabled` function in `torch\csrc\jit\jit_log.cpp`
Therefore, without this pull request, is_enabled does not work properly on Windows.

For more details, please refer to the issue #98145.

Fixes #98145

Pull Request resolved: https://github.com/pytorch/pytorch/pull/98146
Approved by: https://github.com/ezyang
2023-05-04 00:15:28 +00:00
25820b69f6 Revert "[BE] Use data() method when possible as it's safer and more readable (#92755)"
This reverts commit 582485bf0f880de75c7eb36a466562f77e6c64db.

Reverted https://github.com/pytorch/pytorch/pull/92755 on behalf of https://github.com/ezyang due to could have forward fixed but not going to
2023-02-13 21:44:30 +00:00
cyy
fa65ae8f56 cleanup unused include (#93359)
Using `include-what-you-use` tool to find out and remove some unused includes
Pull Request resolved: https://github.com/pytorch/pytorch/pull/93359
Approved by: https://github.com/malfet
2023-02-04 02:15:50 +00:00
582485bf0f [BE] Use data() method when possible as it's safer and more readable (#92755)
Apply clang-tidy readability-data-pointer fixits. This essentially uses the data() method when possible instead of the less readable `&vec[0]` to get the address of the underlying backing implementation. Not only is this more readable, it is safer as it allows you to retrieve the pointer even when the std::vector or std::string is empty without throwing an index error.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/92755
Approved by: https://github.com/ezyang
2023-01-22 20:05:41 +00:00
be71f663f6 StringUtils: Avoid unnecessary allocation in ReplaceAll (#79915)
When the replacement string is smaller than the search string,
`ReplaceAll` can operate in-place on the original string and so
avoid an extra allocation and some string copies.

As suggested in https://github.com/pytorch/pytorch/pull/76976#discussion_r902036461
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79915
Approved by: https://github.com/ezyang
2022-06-27 21:28:04 +00:00
1cf3b24d42 Remove unnecessary allocations in processErrorMsg
`processErrorMsg` uses a table of string constants to be replaced in
the error message. However, this table is non-static so gets
re-constructed from scratch every time. So, I've made it `constexpr`
by using `std::array` instead of `std::vector` and `c10::string_view`
instead of `std::string`.

To support `c10::string_view` I've also updated `c10::ReplaceAll` to
accept string_view arguments, and added a fast-path that also avoids
repeated string searches when no translation is needed.

Using `torch.floor_divide` to benchmark a python warning, I see the
callgrind instruction count fall from 3,806,446 to 703,168 and a 6.5
us time improvement using `%timeit`.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76976

Approved by: https://github.com/swolchok
2022-06-21 00:04:17 +00:00
44cc873fba [PyTorch] Autoformat c10 (#56830)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56830

Opt into formatting on GitHub and format everything. This is a trial run before turning on formatting for more and eventually all of the codebase.

Test Plan: CI

Reviewed By: zertosh

Differential Revision: D27979080

fbshipit-source-id: a80f0c48691c08ae8ca0af06377b87e6a2351151
2021-04-30 21:23:28 -07:00
559e2d8816 Implement optimization bisect (#49031)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/49031

Test Plan: Imported from OSS

Reviewed By: nikithamalgifb

Differential Revision: D25691790

Pulled By: tugsbayasgalan

fbshipit-source-id: a9c4ff1142f8a234a4ef5b1045fae842c82c18bf
2021-01-11 12:25:28 -08:00
15c7486416 Canonicalize includes in c10, and add tests for it (#36299)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/36299

Test Plan: Imported from OSS

Differential Revision: D20943005

Pulled By: ezyang

fbshipit-source-id: 9dd0a58824bd0f1b5ad259942f92954ba1f63eae
2020-04-10 12:07:52 -07:00
73a97387c1 Replace AT_CHECK with TORCH_CHECK [shard 9/10]
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/20435

Reviewed By: jerryzh168

Differential Revision: D15318877

fbshipit-source-id: 4d83571187ea14a604fef83ac355d328b46d93e1
2019-05-15 08:05:59 -07:00
713e706618 Move exception to C10 (#12354)
Summary:
There are still a few work to be done:

- Move logging and unify AT_WARN with LOG(ERROR).
- A few header files are still being plumbed through, need cleaning.
- caffe2::EnforceNotMet aliasing is not done yet.
- need to unify the macros. See c10/util/Exception.h

This is mainly a codemod and not causing functional changes. If you find your job failing and trace back to this diff, usually it can be fixed by the following approaches:

(1) add //caffe2/c10:c10 to your dependency (or transitive dependency).
(2) change objects such as at::Error, at::Optional to the c10 namespace.
(3) change functions to the c10 namespace. Especially, caffe2::MakeString is not overridden by the unified c10::str function. Nothing else changes.

Please kindly consider not reverting this diff - it involves multiple rounds of rebasing and the fix is usually simple. Contact jiayq@ or AI Platform Dev for details.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/12354

Reviewed By: orionr

Differential Revision: D10238910

Pulled By: Yangqing

fbshipit-source-id: 7794d5bf2797ab0ca6ebaccaa2f7ebbd50ff8f32
2018-10-15 13:33:18 -07:00