Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62513

Temp rebalance failure while scaling out from 5 to 6 nodes.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • 7.6.4
    • 7.6.2
    • fts
    • None

    • Enterprise Edition 7.6.2 build 3716 ‧

    Description

      Test Config:

      6 fts nodes - 16vCPU 32 GB disk 200 GB
      3 kv nodes - 8vCPU 32 GB disk 200 GB

      Only 1 FTS Index

      Partitions Partition per node
      30 5

      Steps of the test:
      1. Load 100M data and create FTS Index. 

      2. Wait for indexing complete.

      3. Scale out from 5 to 6 nodes.

      4. Rebalance fails but passes on retry

       

      CPU and memory were well in control

       

      [
        "service_rebalance_failed",
        "fts",
        [
          "agent_died",
          "<0.816.0>",
          [
            "linked_process_died",
            "<0.8035.9>",
            [
              "ns_1@svc-s-node-004.tqvt6a92ifmbdeyd.customsubdomain.nonprod-project-avengers.com",
              [
                "timeout",
                [
                  "gen_server",
                  "call",
                  [
                    "<0.1679.0>",
                    [
                      "call",
                      "ServiceAPI.PrepareTopologyChange",
                      "Fun<json_rpc_connection.0.36915653>",
                      [
                        "timeout=>60000"
                      ]
                    ],
                    "60000"
                  ]
                ]
              ]
            ]
          ]
        ]
      ]  

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sarthak.dua Sarthak Dua
            sarthak.dua Sarthak Dua
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty