Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37998

[Magma] Coredump seen during rebalance in out with hard failover

    XMLWordPrintable

Details

    Description

      Script to Repro

      ./testrunner -i /tmp/ankush.ini  -t rebalance_new.rebalance_in_out.RebalanceInOutTests.test_rebalance_in_out_with_failover,nodes_in=3,nodes_init=3,nodes_out=2,replicas=1,num_items=1000000,rerun=False,magma_storage=True,bucket_storage=magma,infra_log_level=debug,log_level=debug
      

      Steps to Repro

      1  Had a cluster with three nodes
      2 Added three nodes 
      3 Did hard failover, failed over 1 node 
      4. Removed one node
      5. Triggered Rebalance 
      
      

      Below is the backtrace for same

      (gdb) bt
      #0  0x00007f20042b1cbd in cache_bin_dalloc_easy (ptr=<optimized out>, bin_info=<optimized out>, bin=<optimized out>)
          at include/jemalloc/internal/cache_bin.h:126
      #1  tcache_dalloc_small (slow_path=<optimized out>, binind=<optimized out>, ptr=<optimized out>, tcache=<optimized out>,
          tsd=<optimized out>) at include/jemalloc/internal/tcache_inlines.h:178
      #2  arena_dalloc (slow_path=<optimized out>, alloc_ctx=<optimized out>, tcache=<optimized out>, ptr=<optimized out>,
          tsdn=<optimized out>) at include/jemalloc/internal/arena_inlines_b.h:320
      #3  idalloctm (slow_path=<optimized out>, is_internal=<optimized out>, alloc_ctx=<optimized out>, tcache=<optimized out>,
          ptr=<optimized out>, tsdn=<optimized out>) at include/jemalloc/internal/jemalloc_internal_inlines_c.h:118
      #4  ifree (slow_path=<optimized out>, tcache=<optimized out>, ptr=<optimized out>, tsd=<optimized out>) at src/jemalloc.c:2589
      #5  je_dallocx (ptr=ptr@entry=0x7f1fcee09000, flags=<optimized out>) at src/jemalloc.c:3526
      #6  0x00007f2004db134f in cb::_JEArenaMalloc<cb::JEArenaCoreLocalTracker>::free (ptr=ptr@entry=0x7f1fcee09000)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/je_arena_malloc.cc:264
      #7  0x000000000051677a in free (ptr=0x7f1fcee09000)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/include/platform/cb_arena_malloc.h:163
      #8  cb_free (ptr=ptr@entry=0x7f1fcee09000)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/cb_malloc_arena.cc:75
      #9  0x00007f2006c25635 in operator delete (ptr=ptr@entry=0x7f1fcee09000)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/platform/src/global_new_replacement.cc:84
      #10 0x00007f2002b99192 in (anonymous namespace)::run (p=<optimized out>)
          at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/atexit_thread.cc:82
      #11 0x00007f20023debc2 in __nptl_deallocate_tsd () from /usr/lib64/libpthread-2.17.so
      #12 0x00007f20023dedd3 in start_thread () from /usr/lib64/libpthread-2.17.so
      #13 0x00007f200210c21d in clone () from /usr/lib64/libc-2.17.so
      
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ankush.sharma Ankush Sharma
              ankush.sharma Ankush Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty