Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-38633

[System Test]: Rebalance in kv node failed with leader_activities_error

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Major
    • 6.5.1
    • 6.5.1
    • ns_server
    • Centos1

    Description

      Build: 6.5.1-6293 , not seen on 6.5.1-6288

      Test : MH longevity ( -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml )

      Cycle: 3rd

      Day: 1st 

      Test step: Add 1 kv 

      [2020-04-06T22:12:51-07:00, sequoiatools/couchbase-cli:6.5:4bfbb7] server-add -c 172.23.108.103:8091 --server-add https://172.23.106.100 -u Administrator -p password --server-add-username Administrator --server-add-password password --services data
      [2020-04-06T22:13:19-07:00, sequoiatools/couchbase-cli:6.5:2a2c86] rebalance -c 172.23.108.103:8091 -u Administrator -p password
      → 
       
       
      Error occurred on container - sequoiatools/couchbase-cli:6.5:[rebalance -c 172.23.108.103:8091 -u Administrator -p password]
       
       
      docker logs 2a2c86
      docker start 2a2c86
       
       
      *Unable to display progress bar on this os
      JERROR: Rebalance failed. See logs for detailed reason. You can try again.
      [2020-04-06T22:21:09-07:00, sequoiatools/cmd:74b6b3] 60
      
      

      Rebalance failed as 

      Rebalance exited with reason {{badmatch,
      {leader_activities_error,
      {default,{ns_janitor,"default",apply_config}},
      {quorum_lost,
      {lease_lost,'ns_1@172.23.97.122'}}}},
      [{ns_janitor,cleanup_apply_config,4,
      [{file,"src/ns_janitor.erl"},{line,213}]},
      {ns_janitor,
      '-cleanup_membase_bucket/4-fun-0-',4,
      [{file,"src/ns_janitor.erl"},{line,65}]},
      {async,'-async_init/4-fun-1-',3,
      [{file,"src/async.erl"},{line,197}]}]}.
      Rebalance Operation Id = 85dfd6133ebdfb653582c7f4d3a7b64e 

      Attachments

        1. 172.23.108.103-CPU utilization.png
          172.23.108.103-CPU utilization.png
          359 kB
        2. atop.zip
          49.74 MB
        3. screenshot-1.png
          screenshot-1.png
          324 kB
        4. screenshot-2.png
          screenshot-2.png
          243 kB
        5. screenshot-3.png
          screenshot-3.png
          241 kB
        6. screenshot-4.png
          screenshot-4.png
          447 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            vikas.chaudhary Vikas Chaudhary
            vikas.chaudhary Vikas Chaudhary
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty