Echo Writes Code

00000003-open-integrate_core_windows_transcoding.md

Integrate core::windows::transcoding into core::unicode

Explanation

Windows has its own functions for converting between UTF-8 and their native encoding (ostensibly UTF-16, but they don't quite follow the spec apparently). We currently do have a way to call these functions in core::windows::transcoding, but they really should be part of core::unicode (defined only when we're on Windows, obviously).

Furthermore, because these functions were written before core::strings, they still use std::string and std::wstring. We can very easily implement a WindowsEncoding in core::strings that calls the Windows-appropriate functions and uses ::WCHAR code units, and also perform the transcoding on ImmutableSlices / MutableSlices instead of std::strings and std::wstrings.

Task

  • Move everything from core::windows::transcoding into core::unicode
  • Add appropriate #ifdef guards to ensure the Windows functionality is only declared on Windows (it wouldn't compile anywhere else anyway)
  • Convert the functions to use ImmutableSlices as inputs and either MutableSlices or DynamicArrays as outputs (similar to the existing bulk transcoding functions)
  • Speaking of the bulk transcoding functions, rename the Windows functions to correspond to the same naming scheme
  • Add a Windows encoding traits type to core::strings so that we can use String with Windows native strings