Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-10842

cbdocloader can't handle UTF-16 input files created on Windows

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Critical
    • 5.0.0
    • 2.5.1, 3.0
    • tools
    • Security Level: Public
    • Windows 32/64bit

    Description

      When using cbdocloader on Windows, and passing in JSON input files created on Windows in UTF-16 format, cbdocloader fails with "No JSON object could be decoded", as seen below.

      Input files inside the zip are:
      $ file venue_2.json
      venue_2.json: Little-endian UTF-16 Unicode text, with very long lines, with no line terminators

      When running cbdocloader:
      cbdocloader.exe -n localhost:8091 -u Administrator -p [password] -b timbre H:\json\venues.zip

      output

      [2014-04-10 15:02:09,473] - [rest_client] [9308] - INFO - existing buckets : [u'beer-sample', u'gamesim-sample', u'timbre']
      [2014-04-10 15:02:09,473] - [rest_client] [9308] - INFO - found bucket timbre
      No JSON object could be decoded
      No JSON object could be decoded
      No JSON object could be decoded
      done

      If the input file is unzipped, passed through dos2unix, zipped back up again, cbdocloader will process successfully.

      After dos2unix, the input files are:
      $ file venue_1.json
      venue_1.json: ASCII text, with very long lines, with no line terminators

      When running cbdocloader with reformatted input, it works successfully:

      ./cbdocloader -n localhost:8091 -u Administrator -p [password] -b timbre venues_dos2unix.zip
      [2014-04-10 10:24:25,393] - [rest_client] [140735166104336] - INFO - existing buckets : [u'default', u'timbre']
      [2014-04-10 10:24:25,393] - [rest_client] [140735166104336] - INFO - found bucket timbre
      done

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            mikew Mike Wiederhold [X] (Inactive)
            tom.green Tom Green (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty