Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32455

[System test]: Rebalance failed with bad match for cbas

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 6.0.1
    • 6.0.1
    • analytics
    • Untriaged
    • Unknown
    • CX Sprint 133

    Description

      Build: 6.0.1-2014

      Cycle: 4th

      Day : 3rd

      CBAS rebalance is failing for system test

      Test Step : Swap rebalance for analytics

       [2019-01-01T14:32:19-08:00, sequoiatools/couchbase-cli:7e371d] server-add -c 172.23.108.103:8091 --server-add 172.23.104.87:8091 -u Administrator -p password --server-add-username Administrator --server-add-password password --services analytics
      [2019-01-01T14:32:41-08:00, sequoiatools/couchbase-cli:56ed61] rebalance -c 172.23.108.103:8091 --server-remove 172.23.97.239 -u Administrator -p password
       
      Error occurred on container - sequoiatools/couchbase-cli:[rebalance -c 172.23.108.103:8091 --server-remove 172.23.97.239 -u Administrator -p password]
       
      docker logs 56ed61
      docker start 56ed61
       
      *Unable to display progress bar on this os
      JERROR: Rebalance failed. See logs for detailed reason. You can try again.
      [2019-01-01T14:44:34-08:00, sequoiatools/cmd:05514b] 60

      Rebalance exited with reason {service_rebalance_failed,cbas,
      {badmatch,
      {error,
      {bad_nodes,cbas,get_agent,
      [{'ns_1@172.23.104.87',
      {exit,
      {{timeout,
      {gen_server,call,
      [<21559.16733.582>,
      {call,"ServiceAPI.CancelTask",
      #Fun<json_rpc_connection.0.125340786>},
      60000]}},
      {gen_server,call,
      [{'service_agent-cbas',
      'ns_1@172.23.104.87'},
      get_agent,infinity]}}}}]}}}}ns_orchestrator 000ns_1@172.23.108.103 4:16:46 PM   Tue Jan 1, 2019 

      Service 'cbas' exited with status 1. Restarting. Messages:
      at java.base@11.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
      at java.base@11.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
      at java.base@11.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
      at java.base@11.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
      at java.base@11.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.base@11.0.1/java.lang.Thread.run(Thread.java:834)
       
      2019-01-01T20:49:10.906-08:00 INFO CBAS.cbas analytics driver exited; restarting
      2019-01-01T20:49:10.911-08:00 FATA CBAS.cbas my node not found in nodes ([{77b1261e34dfb76c9ecff0d98cd8a704 0 map[cc-http-port:9111 controller-id:13 host:172.23.104.87 num-iodevices:8 starting-partition-id:108]} {79cba4fa74f5ad7b539d06567 ...showns_log 000ns_1@172.23.97.239 8:49:10 PM   Tue Jan 1, 2019 

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            michael.blow Michael Blow
            vikas.chaudhary Vikas Chaudhary
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty