Files
pytorch/c10/util/Unicode.h
Chester Liu 3a943e9f82 Use Unicode friendly API on Win32 in THAllocator (#47905)
Summary:
This replaces the narrow character set APIs with the wide character set ones in `THAllocator.cpp`. This fixes the potential crashes caused by passing non-ASCII characters in `torch::from_file` on Windows.

See: https://github.com/pytorch/pytorch/issues/47422

Pull Request resolved: https://github.com/pytorch/pytorch/pull/47905

Reviewed By: zhangguanheng66

Differential Revision: D25399146

Pulled By: ezyang

fbshipit-source-id: 0a183b65de171c48ed1718fa71e773224eaf196f
2020-12-14 14:24:20 -08:00

30 lines
716 B
C++

#pragma once
#if defined(_WIN32)
#include <string>
#include <c10/util/win32-headers.h>
#include <c10/util/Exception.h>
#endif
namespace c10 {
#if defined(_WIN32)
inline std::wstring u8u16(const std::string& str) {
if (str.empty()) {
return std::wstring();
}
int size_needed = MultiByteToWideChar(
CP_UTF8, 0, str.c_str(), static_cast<int>(str.size()), NULL, 0);
TORCH_CHECK(size_needed > 0, "Error converting the content to Unicode");
std::wstring wstr(size_needed, 0);
MultiByteToWideChar(
CP_UTF8,
0,
str.c_str(),
static_cast<int>(str.size()),
&wstr[0],
size_needed);
return wstr;
}
#endif
}