Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Cheshire-Cat
-
7.0.0-5085
-
Untriaged
-
-
1
-
Unknown
-
KV-Engine CC Final Sprint
Description
- Create a 3 node cluster
- Create required buckets and collections.
- Create 100000000 items sequentially
- Rebalance in with Loading of docs
- Rebalance Out with Loading of docs
- Rebalance In_Out with Loading of docs
- Swap with Loading of docs
- Failover a node and RebalanceOut that node with loading in parallel
Seeing the below message in the logs during last rebalance operation.
Node 172.23.121.124 |
'2021-05-03T11:24:52.288413-07:00 ERROR 726: Exception occurred during packet execution. Closing connection: ThrowExceptionUnderflowPolicy current:9223368318523677506 arg:8244557403892. Cookies: [{"aiostat":"success","connection":"[ {\\"ip\\":\\"127.0.0.1\\",\\"port\\":35209} - {\\"ip\\":\\"127.0.0.1\\",\\"port\\":11209} (System, <ud>@ns_server</ud>) ]","engine_storage":"0x0000000000000000","ewouldblock":false,"packet":{"bodylen":39,"cas":1620066281792274436,"datatype":"raw","extlen":20,"key":"<ud>.random-00040969104</ud>","keylen":19,"magic":"ClientRequest","opaque":347,"opcode":"DCP_EXPIRATION","vbucket":845},"refcount":1}] Exception thrown from: ["#0 /opt/couchbase/bin/memcached() [0x400000+0x2c7543]","#1 /opt/couchbase/bin/memcached() [0x400000+0x8c076]","#2 /opt/couchbase/bin/memcached() [0x400000+0x2fd3b7]","#3 /opt/couchbase/bin/memcached() [0x400000+0x3a08df]","#4 /opt/couchbase/bin/memcached() [0x400000+0x2f522b]","#5 /opt/couchbase/bin/memcached() [0x400000+0x2f560c]","#6 /opt/couchbase/bin/memcached() [0x400000+0x3ff932]","#7 /opt/couchbase/bin/memcached() [0x400000+0x2f2469]","#8 /opt/couchbase/bin/memcached() [0x400000+0x2f3601]","#9 /opt/couchbase/bin/memcached() [0x400000+0x302e1f]","#10 /opt/couchbase/bin/memcached() [0x400000+0x2b9ee8]","#11 /opt/couchbase/bin/memcached() [0x400000+0x4a6a21]","#12 /opt/couchbase/bin/memcached() [0x400000+0x4a7db1]","#13 /opt/couchbase/bin/memcached() [0x400000+0x49b446]","#14 /opt/couchbase/bin/memcached() [0x400000+0x49b683]","#15 /opt/couchbase/bin/memcached() [0x400000+0x49b80a]","#16 /opt/couchbase/bin/memcached() [0x400000+0x25705f]","#17 /opt/couchbase/bin/memcached() [0x400000+0x16c5e4]","#18 /opt/couchbase/bin/memcached() [0x400000+0x1f74a9]","#19 /opt/couchbase/bin/memcached() [0x400000+0x1cbb98]","#20 /opt/couchbase/bin/memcached() [0x400000+0x1b0f48]","#21 /opt/couchbase/bin/memcached() [0x400000+0x1b2777]","#22 /opt/couchbase/bin/memcached() [0x400000+0x1b3391]","#23 /opt/couchbase/bin/../lib/libevent_core-2.1.so.7() [0x7f1954be1000+0xed8e]","#24 /opt/couchbase/bin/../lib/libevent_core-2.1.so.7() [0x7f1954be1000+0x17d01]","#25 /opt/couchbase/bin/../lib/libevent_core-2.1.so.7(event_base_loop+0x3bf) [0x7f1954be1000+0x1854f]","#26 /opt/couchbase/bin/memcached() [0x400000+0x18cc3b]","#27 /opt/couchbase/bin/memcached() [0x400000+0x6b0609]","#28 /lib64/libpthread.so.0() [0x7f19556f3000+0x7ea5]","#29 /lib64/libc.so.6(clone+0x6d) [0x7f1953161000+0xfe8dd]"
|
QE Test |
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/magma_temp_job4.ini bucket_storage=couchstore,bucket_eviction_policy=fullEviction,rerun=False -t volumetests.Magma.volume.SystemTestMagma,nodes_init=3,replicas=1,skip_cleanup=True,num_items=100000000,num_buckets=1,bucket_names=GleamBook,doc_size=64,bucket_type=membase,compression_mode=off,iterations=1,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,maxttl=60,num_collections=20,doc_ops=create:update:delete:expiry,durability=None,pc=1,crashes=10,sdk_client_pool=True'
|