Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-63164

Enforce DocID->Writer mapping in MOI

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • None
    • storage-engine
    • Untriaged
    • 0
    • Unknown

    Description

      The Backindex in MOI uses NodeTables, which are assigned to specific Writers and aren't thread-safe. Each DocID is assigned to a Writer based on the vBucket, calculated by the formula (vBucket % numWriters).

      The canonical vBucket is determined using the formula (((crc32(docid)) >> 16) & 0x7fff) & (numVbuckets - 1).

      1. During updates, MOI uses the vBucket provided by DCP, which may not be canonical.
      2. After an indexer restart, MOI rebuilds the NodeTables assuming all vBuckets are canonical (vbucketFromEntryBytes(...))
      3. If there are non-canonical vBuckets (due to SDK bugs or outdated versions), MOI might use the wrong NodeTable, causing failure during lookup. This may trigger MB-47409.

      Non-canonical vBuckets can be created on SDK level due to a bug or very old SDK version - NCBC-3171, MB-51574. Current approach may also be not compatible with future changes in vBucket calculations.

      Possible solutions:

      1. Always calculate the canonical vBucket within MOI for every update using vbucketFromEntryBytes(...), ignoring the vBucket from DCP.
      2. Use a different hash function to distribute DocIDs across NodeTables.
      3. Persist and keep track of the vBucket for each DocID. This requires managing potential conflicts if different clients use different vBucket calculations for the same DocID.

      Attachments

        Issue Links

          Activity

            People

              srinath.duvuru Srinath Duvuru
              Dmitriy.Kalugin-Balashov Dmitriy Kalugin-Balashov (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                PagerDuty