C++11 - Convert to/from UTF-8/wchar_t
Aug 11, 2016

These functions do work on Linux/Mac and I use them with the Boost.Filesystem API. It is similar to UTF8-CPP and NoWide but not as advanced. It’s more for easily converting between UTF-8 and the native wchar_t type.

  • Always treat std::string as a UTF-8 string
  • Use std::string internally for everything
  • Use wchar_t APIs on Windows whenever possible with widen
  • Convert from std::wstring (use narrow) when saving data to disk
#include <locale>

std::string narrow(const std::wstring & wstr)
{
	std::wstring_convert<std::codecvt_utf8<wchar_t>, wchar_t> convert;
	return convert.to_bytes(wstr);
}

std::wstring widen(const std::string & str)
{
	std::wstring_convert<std::codecvt_utf8<wchar_t>, wchar_t> convert;
	return convert.from_bytes(str);
}

Change the locale inside main() so Boost.Filesystem works correctly on Windows.

#if defined(_WIN32)
	std::locale::global(boost::locale::generator().generate(""));
	boost::filesystem::path::imbue(std::locale());
#endif

Example on Windows with the CreateFileW API:

const std::string path = "my file.txt";
CreateFileW(widen(path).c_str(), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ, nullptr, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
c++ Related
Comments