Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-21786

ForestDB cannot deal with UTF-8 on Windows

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • feature-backlog
    • master
    • forestdb
    • Any version of Windows
    • Untriaged
    • Windows 64-bit
    • No

    Description

      First, to give a hint of the problem, just try compiling ForestDB on windows with the Unicode character set. There are various errors due to misuse of wide strings vs narrow strings on API or types that vary with character set (example LPTSTR, GetFileAttributes, etc). More generally, though, try the following:

      Create a directory with one or more non-ASCII characters in the path.
      Try to open a file in that directory by passing in a UTF-8 encoded string and forestdb will not be able to find it because it doesn't match the code page. This also applies if the path contains characters that cannot be represented by the current code page. Those will simply fail to open with POSIX code 22 (invalid arguments)

      Since Windows file operations are mostly isolated anyway, one fix is to require UTF-8 strings as input (as SQLite does) and then convert them to wide char strings before calling the wide string API. This allows opening even paths with obscure paths (such as the goat emoji) and appending any file IO that is mistakenly not down at that level (such as the calls to 'remove' in filemgr_ops) that make use of file paths.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            tai.tran Tai Tran (Inactive)
            jimb Jim Borden
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty