Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32671

[BP 5.5.x] Cbimport - Key generator does not handle escaped '#' correctly

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 5.5.4, 6.0.2
    • 5.5.2
    • tools
    • Untriaged
    • Unknown

    Description

      Steps to reproduce

      1. Create a Couchbase Server cluster
      2. Create a travel-sample sample bucket
      3. Export the bucket using the following command:

        /opt/couchbase/bin/cbexport json -c couchbase://localhost -u Administrator -p password -b travel-sample -f lines -o output.json
        

      4. Try to import the same dataset back to the bucket with the following command:

        /opt/couchbase/bin/cbimport json -c couchbase://localhost -u Administrator -p password -b travel-sample -d file://output.json -f lines -g 'Matt##%type%##%id%'
        

      5. Then try to import the dataset with a slightly different key generator:

        /opt/couchbase/bin/cbimport json -c couchbase://localhost -u Administrator -p password -b travel-sample -d file://output.json -f lines -g 'Matt##%type%foo##%id%'
        

      Expected Result

      For the first cbimport, the expected result is for this command to succeed and keys such as the following to be generated:

      Matt#airline#10
      

      For the second cbimport, the expected result is for this command to succed and keys such as the following to be generated:

      Matt#airlinefoo#10
      

      Actual Result

      The first cbimport fails with the following error:

      Json import failed: Error in key expression at char 13, invalid generator
      

      The second cbimport does succed, but the keys generated look like:

      Matt##airlinefoo##10
      

      Additional background

      I did some review of the code (https://github.com/couchbase/backup/blob/0019e7f84870b03329ebdeab25d92681fd2c35eb/utils/keygen/key_gen.go), and found a few things to note.

      The codepath where it parses some text (e.g. 'foo') before the double '##' seems to not cause the invalid generator error to be thrown, although it still doesn't seem to correctly substitute this back to a single '#'.

      The case where the 'part' of the generator being parsed starting with a '#' then immediately followed by another '#' is not handled correctly.
      Here (https://github.com/couchbase/backup/blob/0019e7f84870b03329ebdeab25d92681fd2c35eb/utils/keygen/key_gen.go#L144) it will find the next '#' and break out of the function parsing, then it will immediately try to match some regexes to the string '##' which will all fail. Instead there may need to be a special case here to handle that?

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              thuan Thuan Nguyen
              matt.carabine Matt Carabine (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty