Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-29305

Rebalance failed while doc ops are running and KV swap with cbas rebalance in is performed on the cluster with error "The stream closed early because the conn was disconnected".

    XMLWordPrintable

Details

    • Untriaged
    • Unknown

    Description

      Here are the actual steps followed: https://docs.google.com/document/d/1I7yOqAAlRteWmbwS5fpJjqXCcW4n8JmK6fqYGUrKeqg/edit.
      The test is ran at a much lower scale than mentioned in the document. Consider 10M -> 100 docs and 1M update/delete as 10 docs update/delete.
      It is failing at step 50.

      To make it simpler:
      1. There are 3 buckets in the cluster.
      2. Doc ops are performed only in 1 bucket "GleamBookUsers".
      3. There are analytics nodes present in the cluster and cbas bucket and datasets are present.
      4. Make sure create doc ops in progress and do a KV swap rebalance and CBAS node rebalance in in a single go.
      5. Observe that rebalance will fail.

      Here are the logs:
      https://s3.amazonaws.com/bugdb/jira/rebalance_failed/collectinfo-2018-04-18T103925-ns_1%4010.142.164.101.zip
      https://s3.amazonaws.com/bugdb/jira/rebalance_failed/collectinfo-2018-04-18T103925-ns_1%4010.142.164.102.zip
      https://s3.amazonaws.com/bugdb/jira/rebalance_failed/collectinfo-2018-04-18T103925-ns_1%4010.142.164.103.zip
      https://s3.amazonaws.com/bugdb/jira/rebalance_failed/collectinfo-2018-04-18T103925-ns_1%4010.142.164.104.zip
      https://s3.amazonaws.com/bugdb/jira/rebalance_failed/collectinfo-2018-04-18T103925-ns_1%4010.142.164.105.zip
      https://s3.amazonaws.com/bugdb/jira/rebalance_failed/collectinfo-2018-04-18T103925-ns_1%4010.142.164.106.zip

      This issue is consistently reproducible on 5.5.0-2432 build.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty