Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-8747

[Doc'd 2.2] Filter out non-UTF-8 keys and log them

    Details

    • Type: Task
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0
    • Component/s: None
    • Security Level: Public
    • Labels:
      None

      Description

      Copy and paste discussion with Damien

      Junyi, can you write jira ticket that documentation and implementation in this area don't agree? Ideally also fix it and get Filipe to review it, but either way we need the ticket to track the discrepancy so that even when fixed when have a record that some versions don't have the behavior for support and historical purposes.

      -Damien


      Damien Katz
      Couchbase CTO | http://damienkatz.com | 510-421-8914

      Hi Damien,

      As you said, there was a discussion about it and it was determined that upper (ep_engine, couchstore, or couchdb) layer will check non-utf-8 ids and log them before these ids reach XDCR so XDCR won't see any non-utf8 ids. Thus XDCR itself does not validate and log them. Looks like the validation does happen in CouchDB layer (look at couchdb/src/couchdb/couch_doc:json_id()), but I cannot find where the non-utf8 ids are logged.

      If this design does change, seems to me we shall log non-utf8 ids in CouchDB layer, the couch_db:changes_since() function called by XDCR to read changes from CouchDB should filter out and log all non-utf8 ids in CouchDB logs, is that correct?

      Thanks,

      Junyi

        Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          perry Perry Krug added a comment -

          Just a quick bump on this...the RN text seems to indicate the wrong bug number for both this change and the "real" Engineering ticket. Could we track those down and correct?

          Show
          perry Perry Krug added a comment - Just a quick bump on this...the RN text seems to indicate the wrong bug number for both this change and the "real" Engineering ticket. Could we track those down and correct?
          Hide
          kzeller kzeller added a comment - - edited

          So odd because this doc ticket was cloned and references two different engineering tickets:

          https://www.couchbase.com/issues/browse/MB-8427

          and

          http://www.couchbase.com/issues/browse/MB-8732

          True ticket is 8427.

          Removed from comments.

          Verified in RN for 2.2:

          <rnentry type="knownissue">

          <version ver="2.2.0a" />

          <version ver="2.1.0a"/>

          <class id="db"/>

          <issue type="cb" ref="MB-8427"/>

          <rntext>

          <para>
          Any non-UTF-8 characters are not filtered or logged by Couchbase Server. Future releases will address this issue.
          </para>

          </rntext>

          </rnentry>

          <para>
          Any non-UTF-8 characters are not filtered or logged by Couchbase Server. Future releases will address this issue.
          </para>

          </rntext>

          </rnentry>

          Show
          kzeller kzeller added a comment - - edited So odd because this doc ticket was cloned and references two different engineering tickets: https://www.couchbase.com/issues/browse/MB-8427 and http://www.couchbase.com/issues/browse/MB-8732 True ticket is 8427. Removed from comments. Verified in RN for 2.2: <rnentry type="knownissue"> <version ver="2.2.0a" /> <version ver="2.1.0a"/> <class id="db"/> <issue type="cb" ref=" MB-8427 "/> <rntext> <para> Any non-UTF-8 characters are not filtered or logged by Couchbase Server. Future releases will address this issue. </para> </rntext> </rnentry> <para> Any non-UTF-8 characters are not filtered or logged by Couchbase Server. Future releases will address this issue. </para> </rntext> </rnentry>
          Hide
          anil Anil Kumar added a comment -

          it should be

          <para>
          Any non-UTF-8 Keys are not filtered or logged by Couchbase Server. Future releases will address this issue.
          </para>

          Show
          anil Anil Kumar added a comment - it should be <para> Any non-UTF-8 Keys are not filtered or logged by Couchbase Server. Future releases will address this issue. </para>
          Hide
          kzeller kzeller added a comment -

          changed characters to keys

          Show
          kzeller kzeller added a comment - changed characters to keys
          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          Folks, we have 2.5 release notes pointing to this. But frankly it's not clear:

          • whether this was actually fixed or not. I'm seeing commits not merged.
          • if this is not "fixed", it's unclear what's the impact of not having it fixed.

          I'd say that the point having it mentioned in release notes is to warn our users about this. But the way it's done now is IMHO too confusing to be useful. So clearer wording and even internal de-confusion is IMHO required here.

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - Folks, we have 2.5 release notes pointing to this. But frankly it's not clear: whether this was actually fixed or not. I'm seeing commits not merged. if this is not "fixed", it's unclear what's the impact of not having it fixed. I'd say that the point having it mentioned in release notes is to warn our users about this. But the way it's done now is IMHO too confusing to be useful. So clearer wording and even internal de-confusion is IMHO required here.

            People

            • Assignee:
              kzeller kzeller
              Reporter:
              anil Anil Kumar
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes