Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47137

[Magma] ERROR 390: Exception occurred during packet execution. Closing connection: VBucket::processSet: vb:864 expected a complete item but the item is a prepare <ud>cid:0x0:test_docs-27596962</ud> with seqno:11631. Existing prepare has seqno:5927.

    XMLWordPrintable

Details

    Description

      Steps to Recreate:

      1. Create a 4 node cluster and magma bucket with replica=1
      2. Create a create:expiry load with durability=MAJORITY
      3. While load is running in step 2, on all 4 nodes keep klilling memcached 5 times with a wait of 3s in-between each kill.
      4. Wait for the bucket to warmup.
      5. Repeat Step 3, 4 again. This time the error messages is observed in the logs.

      Memcached Logs:
      Error found on Node 172.23.107.33

      ERROR 390: Exception occurred during packet execution. Closing connection: VBucket::processSet: vb:864 expected a complete item but the item is a prepare <ud>cid:0x0:test_docs-27596962</ud> with seqno:11631. Existing prepare has seqno:5927. Cookies: [{"aiostat":"success","connection":"[ {\"ip\":\"127.0.0.1\",\"port\":59449} - {\"ip\":\"127.0.0.1\",\"port\":11209} (System, <ud>@ns_server</ud>) ]","engine_storage":"0x0000000000000000","ewouldblock":false,"packet":{"bodylen":280,"cas":1624677412157456384,"datatype":["JSON"],"extlen":31,"key":"<ud>.test_docs-27596962</ud>","keylen":19,"magic":"ClientRequest","opaque":14,"opcode":"DCP_PREPARE","vbucket":864},"refcount":1}]

      QE-TEST:

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.92477.ini bucket_storage=magma,rerun=false,GROUP=P0;long_kill,randomize_value=true,doc_size=256,bucket_eviction_policy=fullEviction,replicas=1,wait_timeout=600,nodes_init=4,enable_dp=True,collect_pcaps=True,get-cbcollect-info=True,infra_log_level=critical,log_level=error,bucket_storage=magma,upgrade_version=7.1.0-1038 -t storage.magma.magma_crash_recovery.MagmaCrashTests.test_crash_during_ops,num_items=1000000,doc_size=256,sdk_timeout=60,graceful=False,doc_ops=create:expiry,replicas=1,GROUP=P0;long_kill,durability=MAJORITY,multiplier=100'

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-46311 Magma sending stale items in backfill snapshot (GSL/exception thrown in VBucket::processSet with durability)

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ben.huddleston Ben Huddleston
              ankush.sharma Ankush Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty