Details
-
Bug
-
Resolution: Duplicate
-
Major
-
Cheshire-Cat
-
7.0.0-4122
-
Untriaged
-
1
-
Unknown
Description
Steps:
1. Create a 4 node cluster and magma bucket with replica=1
+----------------+----------+--------------+
|
| Nodes | Services | Status |
|
+----------------+----------+--------------+
|
| 172.23.106.218 | kv | Cluster node |
|
| 172.23.106.219 | ['kv'] | <--- IN --- |
|
| 172.23.106.95 | ['kv'] | <--- IN --- |
|
| 172.23.106.99 | ['kv'] | <--- IN --- |
|
+----------------+----------+--------------+
|
2. Create a update:delete:expiry load with durability=PERSIST_TO_MAJORITY for a long duration.
3. While load is running in step 2, on all 4 nodes keep lilling memcached 5 times with a wait of 3s in-between each kill.
4. Wait for the bucket to warmup.
5. Repeat Step 3, 4 again. This time the error messages is observed in the logs.
172.23.106.218:
Found 'exception occurred in runloop' logs - ['2021-01-06T00:43:41.923510-08:00 WARNING 6872: exception occurred in runloop during packet execution. Closing connection: GSL: Precondition failure at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:3229. Cookies: [{"aiostat":"success","connection":"[ {\\"ip\\":\\"127.0.0.1\\",\\"port\\":32802} - {\\"ip\\":\\"127.0.0.1\\",\\"port\\":11209} (<ud>@ns_server</ud>) ]","engine_storage":"0x0000000000000000","ewouldblock":false,"packet":{"bodylen":279,"cas":1609922495994134528,"datatype":["JSON"],"extlen":31,"key":"<ud>.test_docs-1608749</ud>","keylen":18,"magic":"ClientRequest","opaque":39,"opcode":"DCP_PREPARE","vbucket":804},"refcount":1}]\n']
|
QE Test:
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.32045.ini bucket_storage=magma,rerun=false,GROUP=P0;graceful_durability,randomize_value=true,doc_size=256,bucket_eviction_policy=fullEviction,nodes_init=4,graceful=True,upgrade_version=7.0.0-4122 -t magma.magma_crash_recovery.MagmaCrashTests.test_crash_during_recovery,num_items=5000000,doc_size=1024,graceful=True,sdk_timeout=60,doc_ops=update:delete:expiry,replicas=1,durability=PERSIST_TO_MAJORITY,GROUP=P0;graceful_durability'
|
Logs attached!
Attachments
Issue Links
- duplicates
-
MB-46311 Magma sending stale items in backfill snapshot (GSL/exception thrown in VBucket::processSet with durability)
- Closed