Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-26390

Attempting to fetch a document from a corrupted couchstore file silently fails with ENOENT

    XMLWordPrintable

Details

    • Triaged
    • Yes

    Description

      If a couchstore file becomes corrupted such that a document cannot be fetched from disk (returning CHECKSUM_FAIL or any other non-zero status), then instead of logging and returning a error to the user (TMPFAIL), KV-Engine incorrectly returns ENOENT - i.e. document doesn't exist.

      Steps to Reproduce

      1. Create a document with some known value:

      cbc create -U couchbase://localhost:12000/default -u Administrator -P asdasd djr -V abcdefgh
      

      2. Lookup the vBucket ID of the key (390 for this key):

      cbc hash -U couchbase://localhost:12000/default -u Administrator -P asdasd djr
      djr: [vBucket=390, Index=0] Server: localhost:12000, CouchAPI: http://localhost:9500/default
      

      3. Eject that value from memory (so future accesses require background fetch) using the attached script:

      cb_evict_key.py Administrator asdasd default 390 djr
      

      4. Manually corrupt the appropriate couchstore file (390.couch.1), by replacing the value with garbage (this will cause CRC32 to fail):

      perl -pi -e 's/abcdefgh/xxxxxxxx/g' ns_server/data/n_0/data/default/390.couch.1
      

      5. Attempt to read the document:

      cbc cat -U couchbase://localhost:12000/default -u Administrator -P asdasd djr 
      

      Expected Behaviour: TMPFAIL should be returned to the client (asking it to retry):
      djr Temporary failure received from server. Try again later (0xb) and the failure should be logged in memcached.log: Failed background fetch for vb:390, seqno:9

      Actual Behaviour: ENOENT is returned, and no message logged.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            bharath.gp Bharath G P
            drigby Dave Rigby (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty