Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-18871

Rebalance in an FTS node fails [OOM killer]

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • 4.5.0
    • 4.5.0
    • fts
    • None
    • 4.5.0-1910 Enterprise Edition (build-1910)

    Description

      1. Create a cluster for 3 nodes where 1 node is an FTS node.
      2. Create an index and create a few FTS index
      3. Rebalance out the FTS node
      4. Rebalance in the same FTS node as in step3.
      Rebalance fails is shown on UI and following exception in logs. But the node actually gets added to cluster.

      Service 'fts' exited with status 1. Restarting. Messages:  %                  test_1_6b11cae3768f5b5d_f17535e5 0 + 100.0%                      
      2016-03-24T07:52:04.489+00:00 [INFO] ctl/manager, GetTaskList, haveTasksRev: 1080, changed, rv: &service.TaskList{Rev:service.Revision{0x31, 0x30, 0x38, 0x32}, Tasks:[]service.Task{service.Task{Rev:service.Revision{0x31, 0x30, 0x38, 0x31}, ID:"rebalance:52683c46392501cfe068809cffc1e086", Type:"task-rebalance", Status:"task-running", IsCancelable:true, Progress:15.664804469273744, DetailedProgress:map[service.NodeID]float64{}, Description:"topology change", ErrorMessage:"", Extra:map[string]interface {}{"topologyChange":service.TopologyChange{ID:"52683c46392501cfe068809cffc1e086", CurrentTopologyRev:service.Revision(nil), Type:"topology-change-rebalance", KeepNodes:[]struct { NodeInfo service.NodeInfo "json:\"nodeInfo\""; RecoveryType service.RecoveryType "json:\"recoveryType\"" }{struct { NodeInfo service.NodeInfo "json:\"nodeInfo\""; RecoveryType service.RecoveryType "json:\"recoveryType\"" }{NodeInfo:service.NodeInfo{NodeID:"f4294a8dc1fbb1b4a3be89278a03b30e", Priority:0, Opaque:interface {}(nil)}, RecoveryType:"recovery-full"}}, EjectNodes:[]service.NodeInfo(nil)}}}}}
      2016-03-24T07:52:07.678+00:00 [INFO] ctl/manager, revNum: 1083, progress: 15.664804
      2016-03-24T07:52:07.907+00:00 [INFO] ctl/manager, GetTaskList, haveTasksRev: 1082, changed, rv: &service.TaskList{Rev:service.Revision{0x31, 0x30, 0x38, 0x34}, Tasks:[]service.Task{service.Task{Rev:service.Revision{0x31, 0x30, 0x38, 0x33}, ID:"rebalance:52683c46392501cfe068809cffc1e086", Type:"task-rebalance", Status:"task-running", IsCancelable:true, Progress:15.664804469273744, DetailedProgress:map[service.NodeID]float64{}, Description:"topology change", ErrorMessage:"", Extra:map[string]interface {}{"topologyChange":service.TopologyChange{ID:"52683c46392501cfe068809cffc1e086", CurrentTopologyRev:service.Revision(nil), Type:"topology-change-rebalance", KeepNodes:[]struct { NodeInfo service.NodeInfo "json:\"nodeInfo\""; RecoveryType service.RecoveryType "json:\"recoveryType\"" }{struct { NodeInfo service.NodeInfo "json:\"nodeInfo\""; RecoveryType service.RecoveryType "json:\"recoveryType\"" }{NodeInfo:service.NodeInfo{NodeID:"f4294a8dc1fbb1b4a3be89278a03b30e", Priority:0, Opaque:interface {}(nil)}, RecoveryType:"recovery-full"}}, EjectNodes:[]service.NodeInfo(nil)}}}}}
      [goport] 2016/03/24 07:52:08 /opt/couchbase/bin/cbft terminated: signal: killed
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            steve Steve Yen
            ritam.sharma Ritam Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty