Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-38521

[System Test]: Server delta Recovery resulting in crash

    XMLWordPrintable

Details

    • Untriaged
    • Yes

    Description

      Build : 6.5.1-6288
      Test : -test tests/eventing/mad-hatter/test_eventing_rebalance.yml -scope tests/eventing/mad-hatter/scope_eventing_rebalance.yml
      Scale : 2
      Iteration : 1st (day 1)

      Test Step:

      [2020-03-31T12:08:50-07:00, sequoiatools/couchbase-cli:6.5:f7b7a7] recovery -c 172.23.104.16:8091 --server-recovery 172.23.104.18:8091 --recovery-type delta -u Administrator -p password
      		 
      [2020-03-31T12:08:57-07:00, sequoiatools/couchbase-cli:6.5:90886b] rebalance -c 172.23.104.16:8091 -u Administrator -p password
      		 
      [2020-03-31T12:16:13-07:00, sequoiatools/cmd:7050eb] 60

      Crash on 172.23.104.18:

      [ns_server:info,2020-03-31T12:08:56.822-07:00,babysitter_of_ns_1@cb.local:<0.11213.0>:ns_port_server:log:224]memcached<0.11213.0>: 2020-03-31T12:08:56.621533-07:00 CRITICAL Breakpad caught a crash (Couchbase version 6.5.1-6288). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/2e8c2618-cf61-6297-7578c9bd-5f255526.dmp before terminating.
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621566-07:00 CRITICAL Stack backtrace of crashed thread:
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621858-07:00 CRITICAL     /opt/couchbase/bin/memcached() [0x400000+0x1329ed]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621891-07:00 CRITICAL     /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ce) [0x400000+0x14aaae]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621946-07:00 CRITICAL     /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0x94) [0x400000+0x14adc4]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621958-07:00 CRITICAL     /lib64/libpthread.so.0() [0x7f1418892000+0xf5e0]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621969-07:00 CRITICAL     /lib64/libpthread.so.0(pthread_mutex_lock+0) [0x7f1418892000+0x9c30]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621981-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0x7badf]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621989-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0xabc23]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.621997-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0x701ed]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622003-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0x73ec7]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622057-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0xae8e8]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622073-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0x8e400]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622078-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0x9146a]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622086-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7f1413aaa000+0xfd6aa]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622098-07:00 CRITICAL     /opt/couchbase/bin/memcached() [0x400000+0x254e5]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622109-07:00 CRITICAL     /opt/couchbase/bin/memcached() [0x400000+0x25b69]
      		 
      memcached<0.11213.0>: 2020-03-31T12:08:56.622118-07:00 CRITICAL     /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0(_ZN9Couchbase6Thread12thread_entryEv+0xf) [0x7f141b076000+0x147bf]

      Logs:

      cbcollect logs:

      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.16.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.17.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.18.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.19.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.21.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.23.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.93.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.104.94.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.96.30.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1585686976/collectinfo-2020-03-31T203619-ns_1%40172.23.96.96.zip

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            girish.benakappa Girish Benakappa
            girish.benakappa Girish Benakappa
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty