Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47104

[Magma] - "Service 'memcached' exited with status 137" seen post rebalance

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Major
    • None
    • 7.1.0
    • couchbase-bucket
    • Enterprise Edition 7.1.0 build 1037 ‧ IPv4 © 2021 Couchbase, Inc.
    • Untriaged
    • Centos 64-bit
    • 1
    • No

    Description

      Script to Repro

      ./sequoia -client 172.23.96.162:2375 -provider file:centos_third_cluster.yml -test tests/rebalance/test_allRebalance_collections.yml -scope tests/rebalance/scope_rebalance_collection.yml -scale 3 -repeat 0 -log_level 0 -version 7.1.0-1037 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=0 -show_topology=true
      

      Was trying to setup magma component level system test and noticed few memcached exits with barely any data post rebalance. 2 different kind of exits were seen. Possibly related to each other.

      ns_1@172.23.104.15 10:06:20 PM 24 Jun, 2021

      Service 'memcached' exited with status 137. Restarting. Messages:
      WARNING: Logging before InitGoogleLogging() is written to STDERR
      W0624 09:29:44.592945 118642 HazptrDomain.h:671] Using the default inline executor for asynchronous reclamation may be susceptible to deadlock if the current thread happens to hold a resource needed by the deleter of a reclaimable object
      W0624 14:13:57.881852 118637 HazptrDomain.h:671] Using the default inline executor for asynchronous reclamation may be susceptible to deadlock if the current thread happens to hold a resource needed by the deleter of a reclaimable object
      W0624 18:55:09.168877 118637 HazptrDomain.h:671] Using the default inline executor for asynchronous reclamation may be susceptible to deadlock if the current thread happens to hold a resource needed by the deleter of a reclaimable object
      

      ns_1@172.23.104.15 10:09:35 PM 24 Jun, 2021

      Service 'memcached' exited with status 137. Restarting. Messages:
      --65866-- When reading debug info from /data/bucket4/magma.6/wal/wal.2:
      --65866-- can't read file to inspect ELF header
      --65866-- WARNING: Serious error when reading debug info
      --65866-- When reading debug info from /data/bucket4/magma.7/wal/wal.2:
      --65866-- can't read file to inspect ELF header
      --65866-- WARNING: Serious error when reading debug info
      --65866-- When reading debug info from /data/bucket4/magma.7/wal/wal.2:
      --65866-- can't read file to inspect ELF header
      --65866-- WARNING: Serious error when reading debug info
      --65866-- When reading debug info from /data/bucket4/magma.7/wal/wal.2:
      --65866-- can't read file to inspect ELF header
      WARNING: Logging before InitGoogleLogging() is written to STDERR
      W0624 22:06:54.668663 65902 HazptrDomain.h:671] Using the default inline executor for asynchronous reclamation may be susceptible to deadlock if the current thread happens to hold a resource needed by the deleter of a reclaimable object
      /opt/couchbase/bin/memcached: line 3: 65866 Killed valgrind --tool=massif /opt/couchbase/bin/memcached_proc $@
      

      cbcollect_info attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty