Loading...

Details

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: 7.1.0
Affects Version/s: 7.1.0
Component/s: couchbase-bucket
Labels:
Environment:
Enterprise Edition 7.1.0 build 2298

Triage:
Untriaged
Operating System:
Centos 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.104.249.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.105.0.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.105.38.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.105.39.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.106.37.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.106.54.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.107.236.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.107.25.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.132.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.134.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.136.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.138.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.139.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.141.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.142.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.143.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.144.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.145.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.146.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.148.zip

Show
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.104.249.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.105.0.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.105.38.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.105.39.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.106.37.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.106.54.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.107.236.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.107.25.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.132.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.134.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.136.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.138.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.139.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.141.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.142.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.143.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.144.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.145.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.146.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1644833738/collectinfo-2022-02-14T101540-ns_1%40172.23.108.148.zip
Epic Link:
KV: Magma
Story Points:
1
Is this a Regression?:
Yes
Sprint:
KV 2022-Feb

Description

Test -

-test tests/integration/neo/test_neo_magma_milestone4.yml -scope tests/integration/neo/scope_neo_magma.yml

Day - 1
Cycle - 1
Scale - 3

grep CRITICAL on 172.23.108.143

grep "CRITICAL" memcached.log

2022-02-14T00:28:36.535672-08:00 CRITICAL Detected previous crash

2022-02-14T00:28:36.535746-08:00 CRITICAL Breakpad caught a crash (Couchbase version 7.1.0-2298). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/038214a8-6c19-4944-41357fb6-2bb132bc.dmp before terminating.

2022-02-14T00:28:36.535748-08:00 CRITICAL Stack backtrace of crashed thread:

2022-02-14T00:28:36.535750-08:00 CRITICAL    #0  /opt/couchbase/bin/memcached() [0x400000+0x736228]

2022-02-14T00:28:36.535752-08:00 CRITICAL    #1  /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ea) [0x400000+0x7879fa]

2022-02-14T00:28:36.535753-08:00 CRITICAL    #2  /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xb8) [0x400000+0x787d38]

2022-02-14T00:28:36.535755-08:00 CRITICAL    #3  /lib64/libpthread.so.0() [0x7f50b2b24000+0xf630]

2022-02-14T00:28:36.535756-08:00 CRITICAL    #4  /opt/couchbase/bin/../lib/libevent_core-2.1.so.7(bufferevent_incref_and_lock_+0x1) [0x7f50b1ddf000+0xe6a1]

2022-02-14T00:28:36.535758-08:00 CRITICAL    #5  /opt/couchbase/bin/../lib/libevent_core-2.1.so.7(bufferevent_trigger+0x22) [0x7f50b1ddf000+0xead2]

2022-02-14T00:28:36.535759-08:00 CRITICAL    #6  /opt/couchbase/bin/memcached() [0x400000+0x1b2959]

2022-02-14T00:28:36.535760-08:00 CRITICAL    #7  /opt/couchbase/bin/memcached() [0x400000+0x814f7b]

2022-02-14T00:28:36.535761-08:00 CRITICAL    #8  /opt/couchbase/bin/memcached() [0x400000+0x80e9b8]

2022-02-14T00:28:36.535762-08:00 CRITICAL    #9  /opt/couchbase/bin/memcached() [0x400000+0x810c79]

2022-02-14T00:28:36.535763-08:00 CRITICAL    #10 /opt/couchbase/bin/memcached() [0x400000+0x811866]

2022-02-14T00:28:36.535764-08:00 CRITICAL    #11 /opt/couchbase/bin/memcached() [0x400000+0x813226]

2022-02-14T00:28:36.535766-08:00 CRITICAL    #12 /opt/couchbase/bin/memcached() [0x400000+0x1b2209]

2022-02-14T00:28:36.535767-08:00 CRITICAL    #13 /opt/couchbase/bin/memcached() [0x400000+0x79c119]

2022-02-14T00:28:36.535768-08:00 CRITICAL    #14 /lib64/libpthread.so.0() [0x7f50b2b24000+0x7ea5]

2022-02-14T00:28:36.535770-08:00 CRITICAL    #15 /lib64/libc.so.6(clone+0x6d) [0x7f50b036e000+0xfeb0d]

Service exit accompanying the crash
2022-02-14T00:28:36.359-08:00, ns_log:0:info:message(ns_1@172.23.108.143) - Service 'memcached' exited with status 139. Restarting.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

bt_full.txt
736 kB
14/Feb/22 6:16 AM

Issue Links

is duplicated by

MB-50998 [System Test] Index rebalance failed due to error - "Error Connecting KV 127.0.0.1:8091 Err MCResponse status=KEY_ENOENT"

Closed

MB-51057 [magma]: memcached crashed during replica update on xdcr source cluster.

Closed

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

[System Test][Magma] Memcached crash along with service exit observed in longevity

Details

Description

Attachments

Attachments

Issue Links

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty