Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-8427

Filter out non-UTF-8 keys and log them

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Won't Fix
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0
    • Component/s: storage-engine, XDCR
    • Security Level: Public
    • Labels:
      None

      Description

      Copy and paste discussion with Damien

      Junyi, can you write jira ticket that documentation and implementation in this area don't agree? Ideally also fix it and get Filipe to review it, but either way we need the ticket to track the discrepancy so that even when fixed when have a record that some versions don't have the behavior for support and historical purposes.

      -Damien


      Damien Katz
      Couchbase CTO | http://damienkatz.com | 510-421-8914

      Hi Damien,

      As you said, there was a discussion about it and it was determined that upper (ep_engine, couchstore, or couchdb) layer will check non-utf-8 ids and log them before these ids reach XDCR so XDCR won't see any non-utf8 ids. Thus XDCR itself does not validate and log them. Looks like the validation does happen in CouchDB layer (look at couchdb/src/couchdb/couch_doc:json_id()), but I cannot find where the non-utf8 ids are logged.

      If this design does change, seems to me we shall log non-utf8 ids in CouchDB layer, the couch_db:changes_since() function called by XDCR to read changes from CouchDB should filter out and log all non-utf8 ids in CouchDB logs, is that correct?

      Thanks,

      Junyi

        Issue Links

          Activity

          junyi Junyi Xie (Inactive) created issue -
          dipti Dipti Borkar made changes -
          Field Original Value New Value
          Assignee Junyi Xie [ junyi ] Ravi Mayuram [ rmayuram ]
          maria Maria McDuff (Inactive) made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          rmayuram Ravi Mayuram made changes -
          Assignee Ravi Mayuram [ rmayuram ] Junyi Xie [ junyi ]
          junyi Junyi Xie (Inactive) made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          junyi Junyi Xie (Inactive) added a comment -

          http://review.couchbase.org/#/c/27866/

          fix on gerrit pending review.

          Show
          junyi Junyi Xie (Inactive) added a comment - http://review.couchbase.org/#/c/27866/ fix on gerrit pending review.
          anil Anil Kumar made changes -
          Link This issue blocks MB-8747 [ MB-8747 ]
          Hide
          junyi Junyi Xie (Inactive) added a comment -

          Please see discussion at MB-8747

          put the commit on gerrit on hold waiting for further decision from PM.

          Show
          junyi Junyi Xie (Inactive) added a comment - Please see discussion at MB-8747 put the commit on gerrit on hold waiting for further decision from PM.
          junyi Junyi Xie (Inactive) made changes -
          Assignee Junyi Xie [ junyi ] Dipti Borkar [ dipti ]
          junyi Junyi Xie (Inactive) made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          junyi Junyi Xie (Inactive) made changes -
          Comment [ not a xdcr issue per discussion with Ravi and Dipti

          if we change decision later, we can reopen it or create another ticket.

          ]
          Hide
          maria Maria McDuff (Inactive) added a comment -

          already done.

          Show
          maria Maria McDuff (Inactive) added a comment - already done.
          maria Maria McDuff (Inactive) made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              dipti Dipti Borkar
              Reporter:
              junyi Junyi Xie (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes