Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-35316

[System Test] Memcached crashes with status 137

    XMLWordPrintable

Details

    Description

      Build : 6.5.0-3883
      Test : -test tests/2i/test_idx_rebalance_replica_vulcan_kv_opt.yml -scope tests/2i/scope_idx_rebalance_replica_vulcan.yml
      Scale : 2
      Iteration : 6th

      In the GSI component test, we are seeing a memcached crash on 172.23.104.18. Following is seen in the info.log

      [ns_server:warn,2019-07-28T10:13:35.097-07:00,ns_1@172.23.104.18:<0.2657.330>:ns_memcached:connect:1095]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
      		 
      [ns_server:warn,2019-07-28T10:13:35.106-07:00,ns_1@172.23.104.18:<0.2677.330>:ns_memcached:connect:1095]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
      		 
      [ns_server:warn,2019-07-28T10:13:35.143-07:00,ns_1@172.23.104.18:<0.2691.330>:ns_memcached:connect:1095]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
      		 
      [ns_server:warn,2019-07-28T10:13:35.143-07:00,ns_1@172.23.104.18:<0.5797.329>:ns_memcached:connect:1095]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
      		 
      [user:info,2019-07-28T10:13:35.413-07:00,ns_1@172.23.104.18:<0.9642.0>:ns_log:crash_consumption_loop:69]Service 'memcached' exited with status 137. Restarting. Messages:
      		 
      2019-07-28T10:13:24.316728-07:00 WARNING (other-2) Slow runtime for 'Item pager on vb:1024' on thread nonIO_worker_2: 898 ms
      		 
      2019-07-28T10:13:24.351512-07:00 WARNING (other-1) Slow runtime for 'Item pager on vb:1024' on thread nonIO_worker_1: 934 ms
      		 
      2019-07-28T10:13:24.369457-07:00 WARNING (other-3) Slow scheduling for NON_IO task 'Removing closed unreferenced checkpoints from memory' on thread nonIO_worker_1. Schedule overhead: 1000 ms
      		 
      2019-07-28T10:13:24.375171-07:00 WARNING (other-1) Slow scheduling for NON_IO task 'Item Compressor' on thread nonIO_worker_2. Schedule overhead: 1009 ms
      		 
      2019-07-28T10:13:24.383465-07:00 WARNING (other-2) Slow scheduling for NON_IO task 'Item Compressor' on thread nonIO_worker_2. Schedule overhead: 1016 ms
      		 
      2019-07-28T10:13:24.383499-07:00 WARNING (other-3) Slow scheduling for NON_IO task 'Item Compressor' on thread nonIO_worker_1. Schedule overhead: 1016 ms
      		 
      2019-07-28T10:13:26.165658-07:00 WARNING (other-1) Slow runtime for 'Checkpoint Remover on vb:749' on thread nonIO_worker_2: 61 ms

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-35252 Graceful Failover rebalance time increased by ~180% in build 6.5.0-3790

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              mihir.kamdar Mihir Kamdar (Inactive)
              mihir.kamdar Mihir Kamdar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty