Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-35792

memcached crashes seen in the weekly runs during autofailover tests

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 6.5.0
    • couchbase-bucket
    • None
    • 6.5.0-4169

    Description

      Script to Repro

      ./testrunner -i /tmp/testexec.3938.ini get-cbcollect-info=True,durability=MAJORITY_AND_PERSIST_ON_MASTER,num_items=100000,GROUP=P0;durability -t failover.AutoFailoverTests.AutoFailoverTests.test_autofailover_after_rebalance,timeout=5,num_node_failures=1,nodes_in=1,nodes_out=1,replicas=2,failover_orchestrator=True,failover_action=restart_server,nodes_init=4,can_abort_rebalance=False,GROUP=P0;durability
      

      This is an autofailover tests with restart_server as the failover action.
      I noticed few memcached crashes with following messages.

      Service 'memcached' exited with status 134. Restarting. Messages:\n2019-09-02T21:57:05.418173-07:00 CRITICAL     /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7ff3a8984000+0x8efd1]\n2019-09-02T21:57:05.418187-07:00 CRITICAL     /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7ff3a8984000+0x8f213]\n2019-09-02T21:57:05.418198-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7ff3a376e000+0xd4308]\n2019-09-02T21:57:05.418206-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7ff3a376e000+0xe83ef]\n2019-09-02T21:57:05.418218-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7ff3a376e000+0x136235]\n2019-09-02T21:57:05.418225-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7ff3a376e000+0x1366fd]\n2019-09-02T21:57:05.418232-07:00 CRITICAL     /opt/couchbase/bin/../lib/../lib/ep.so() [0x7ff3a376e000+0x12fc6c]\n2019-09-02T21:57:05.418237-07:00 CRITICAL     /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0() [0x7ff3aa830000+0x8f27]\n2019-09-02T21:57:05.418243-07:00 CRITICAL     /lib64/libpthread.so.0() [0x7ff3a824f000+0x7dd5]\n2019-09-02T21:57:05.418273-07:00 CRITICAL     /lib64/libc.so.6(clone+0x6d) [0x7ff3a7e82000+0xfdead]
      

      PFA cbcollect_info.
      Ran it through gdb, not seeing anything useful though. Not sure if I am missing something.

      (gdb) bt
      #0  0x00007f65c5dbe207 in __gconv_transform_internal_ucs2reverse () from /usr/lib64/libc-2.17.so
      #1  0x00007f65ba124000 in ?? ()
      #2  0x00007f65c5e03dc3 in wprintf () from /usr/lib64/libc-2.17.so
      #3  0x00007f65ba043000 in ?? ()
      #4  0x0000000000000000 in ?? ()
      (gdb)
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Balakumaran.Gopal Balakumaran Gopal
              Balakumaran.Gopal Balakumaran Gopal
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty