Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41257

[Jepsen] Linearizability failure during disk failure test

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • 7.0.0
    • Cheshire-Cat
    • couchbase-bucket
    • Ubuntu1604
    • Triaged
    • 1
    • Unknown
    • KV-Engine 2021-Jan, KV-Engine 2021-March

    Description

      Linearizability failure found during jepsen-post-commit-2240 while running test lein trampoline run test --nodes-file ./nodes --username vagrant --ssh-private-key ./resources/vagrantkey --install-path /home/couchbase/jenkins/workspace/kv-engine-jepsen-post-commit/install --workload=disk-failure --manipulate-disks --replicas=1 --durability=0:0:0:100 --recovery-type=delta --eviction-policy=full --cas --hashdump --enable-memcached-debug-log-level --enable-tcp-capture --autofailover-timeout=30.

      Logs attached here:

      Attachments

        1. 20200831T220342.000-0700.zip
          44.61 MB
        2. 47f397db-2f20-445d-e8ec14b1-96b621dd.dmp
          508 kB
        3. linear.svg
          172 kB
        4. register-13.png
          register-13.png
          66 kB
        5. register-13-flow.png
          register-13-flow.png
          56 kB

        Issue Links

          depends on

          Improvement - An improvement or enhancement to an existing feature or task. MB-20294 Loss of data on replicas when active vbucket files are deleted and warmup completes

          • Critical - Crashes, loss of data, severe memory leak.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. MB-31711 Primary crash and restart within auto-failover window should not lose durable write

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MB-45721 [Couchstore]: Warmup::populateVBucketMap() flush state failed for vb:251 highSeqno:5071, write traffic will be disabled for this node.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          For Gerrit Dashboard: MB-41257
          # Subject Branch Project Status CR V
          144385,3 MB-41257: Check for disk failure master kv_engine Status: ABANDONED 0 +1
          144737,8 MB-41257: Don't enable traffic if Warmup sees failure from KVStore master kv_engine Status: ABANDONED 0 0
          145970,3 MB-41257: Add kill and disk failure test master jepsen.couchbase Status: MERGED +2 +1
          146316,2 Revert "MB-41257: Update configs to run kill-n-disk-failure workload" master jepsen.couchbase Status: MERGED +2 +1
          146330,2 MB-41257: Update configs to run kill-n-disk-failure workload master jepsen.couchbase Status: MERGED +2 +1
          146837,2 MB-41257: Re-Add Update configs to run kill-n-disk-failure workload master jepsen.couchbase Status: MERGED +2 +1
          146952,1 MB-41257: Abort warmup if we fail to read persisted stats master kv_engine Status: ABANDONED -1 +1
          146961,9 MB-41257: Abort warmup if we fail to make KVFileHandle to vbucket file master kv_engine Status: MERGED +2 +1
          147058,11 MB-41257: Abort warmup if we're unable to scan the KVStore for prepares master kv_engine Status: MERGED +2 +1
          147349,17 MB-41257: Don't enable traffic if Warmup can't write vb state to disk master kv_engine Status: MERGED +2 +1
          147724,9 MB-41257: Abort warmup if we're unable to get collection states master kv_engine Status: MERGED +2 +1
          147744,5 MB-41257: Reset vbucket to avoid bad state of PDM master kv_engine Status: MERGED +2 +1
          149724,5 MB-41257: Make WindowsFileOps::open() use oflags to decide R/W mode master couchstore Status: MERGED +2 +1

          Activity

            People

              asad.zaidi Asad Zaidi (Inactive)
              richard.demellow Richard deMellow
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty