Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Cheshire-Cat
-
Centos 7 64 bit; Couchbase EE 7.0.0-4325
-
Untriaged
-
Centos 64-bit
-
-
1
-
Yes
Description
Script to Repo:
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.21631.ini GROUP=rebalance_with_collection_crud_durability_MAJORITY,rerun=False,upgrade_version=7.0.0-4325 -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_rebalance_in,nodes_init=3,nodes_in=2,override_spec_params=durability;replicas,durability=MAJORITY,replicas=2,bucket_spec=multi_bucket.buckets_for_rebalance_tests_more_collections,data_load_spec=volume_test_load_with_CRUD_on_collections,data_load_stage=during,quota_percent=80,GROUP=rebalance_with_collection_crud_durability_MAJORITY' |
Steps to Reproduce:
1. Create a 3 node cluster
2021-01-27 21:54:32,744 | test | INFO | pool-1-thread-7 | [table_view:display:72] Rebalance Overview
------------------------------------
| Nodes | Services | Status |
------------------------------------
| 172.23.105.122 | kv | Cluster node |
| 172.23.105.10 | None | <--- IN — |
| 172.23.105.137 | None | <--- IN — |
------------------------------------
2. Create buckets and create collections
2021-01-27 21:57:09,529 | test | INFO | MainThread | [table_view:display:72] Bucket statistics |
+---------+-----------+----------+------------+-----+--------+------------+-----------+-----------+
|
| Bucket | Type | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used |
|
+---------+-----------+----------+------------+-----+--------+------------+-----------+-----------+
|
| bucket1 | couchbase | 2 | none | 0 | 3000 | 629145600 | 161977592 | 296765954 | |
| bucket2 | ephemeral | 2 | none | 0 | 3000 | 629145600 | 239703112 | 102 | |
| default | couchbase | 2 | none | 0 | 500000 | 6291456000 | 540219240 | 413180251 | |
+---------+-----------+----------+------------+-----+--------+------------+-----------+-----------+
|
|
3. Rebalance-in with crud on collections with durability load Majority
This rebalance fails and minidump is seen
2021-01-27 21:59:34,464 | test | CRITICAL | MainThread | [basetestcase:check_coredump_exist:728] 172.23.105.122: 1 core dump seen |
on .122 memcached.log
2021-01-27T21:58:57.936507-08:00 CRITICAL *** Fatal error encountered during exception handling *** |
2021-01-27T21:58:57.936550-08:00 CRITICAL Caught unhandled std::exception-derived exception. what(): Collections::VB::Manifest::verifyFlatbuffersData: getCreateEventData data invalid, ptr:0x7f01ab93e679, size:32514 |
2021-01-27T21:58:58.045168-08:00 CRITICAL Breakpad caught a crash (Couchbase version 7.0.0-4325). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/379b3679-489b-43b0-6d344683-76ca816e.dmp before terminating. |
2021-01-27T21:58:58.045180-08:00 CRITICAL Stack backtrace of crashed thread: |
2021-01-27T21:58:58.045332-08:00 CRITICAL /opt/couchbase/bin/memcached() [0x400000+0x145bbd] |
2021-01-27T21:58:58.045343-08:00 CRITICAL /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ea) [0x400000+0x15b3fa] |
2021-01-27T21:58:58.045350-08:00 CRITICAL /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xb8) [0x400000+0x15b738] |
2021-01-27T21:58:58.045356-08:00 CRITICAL /lib64/libpthread.so.0() [0x7f023af4a000+0xf370] |
2021-01-27T21:58:58.045374-08:00 CRITICAL /lib64/libc.so.6(gsignal+0x37) [0x7f023ab89000+0x351d7] |
2021-01-27T21:58:58.045391-08:00 CRITICAL /lib64/libc.so.6(abort+0x148) [0x7f023ab89000+0x368c8] |
2021-01-27T21:58:58.045418-08:00 CRITICAL /opt/couchbase/bin/../lib/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x125) [0x7f023b67f000+0x91195] |
2021-01-27T21:58:58.045428-08:00 CRITICAL /opt/couchbase/bin/memcached() [0x400000+0x155632] |
2021-01-27T21:58:58.045442-08:00 CRITICAL /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f023b67f000+0x8ef86] |
2021-01-27T21:58:58.045456-08:00 CRITICAL /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f023b67f000+0x8efd1] |
2021-01-27T21:58:58.045465-08:00 CRITICAL /opt/couchbase/bin/../lib/libep.so() [0x7f023efa9000+0x16dc33] |
2021-01-27T21:58:58.045471-08:00 CRITICAL /opt/couchbase/bin/../lib/libep.so() [0x7f023efa9000+0x167c92] |
2021-01-27T21:58:58.045480-08:00 CRITICAL /opt/couchbase/bin/../lib/libep.so() [0x7f023efa9000+0x2e71d6] |
2021-01-27T21:58:58.045487-08:00 CRITICAL /opt/couchbase/bin/../lib/libep.so() [0x7f023efa9000+0x2cf6ca] |
2021-01-27T21:58:58.045494-08:00 CRITICAL /opt/couchbase/bin/../lib/libep.so() [0x7f023efa9000+0x2ea2f9] |
2021-01-27T21:58:58.045500-08:00 CRITICAL /opt/couchbase/bin/../lib/libep.so() [0x7f023efa9000+0x1660d3] |
2021-01-27T21:58:58.045522-08:00 CRITICAL /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f023b67f000+0xb9dcf] |
2021-01-27T21:58:58.045527-08:00 CRITICAL /lib64/libpthread.so.0() [0x7f023af4a000+0x7dc5] |
2021-01-27T21:58:58.045552-08:00 CRITICAL /lib64/libc.so.6(clone+0x6d) [0x7f023ab89000+0xf776d] |
Will attach logs soon
Attachments
Issue Links
- relates to
-
-
- Closed
-