Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-30135

Data lost from cbas forever when cbas node(cc/nc) is failed over and secondary indexes are present. Ingestion doesn't work even after disconnect and connect.

    XMLWordPrintable

Details

    • Untriaged
    • Unknown

    Description

      Steps:
      1. Initialize a cluster with 1 KV and 4 cbas nodes.
      2. Create a default bucket.
      3. Setup cbas and create secondary indexes.

      create bucket default_bucket with {"name":"default"};
      create shadow dataset default_ds on default_bucket;
      create index NoName on default_ds(profession:string,number:bigint);
      select * from Metadata.`Index` where DatasetName='default_ds' and IsPrimary=False
      connect bucket default_bucket;
      

      4. Load 20k docs now in default bucket.
      5. Failover CC and rebalance it out.
      6. Check for docs in CBAS after rebalance is completed. It is 0.
      7. Disconnect and reconnect cbas buckets and check if data ingestion starts. It doesn't.

      Logs:
      https://s3.amazonaws.com/bugdb/jira/FailoverCC_SecondaryIndexes/collectinfo-2018-06-15T131515-ns_1%40172.23.96.55.zip
      https://s3.amazonaws.com/bugdb/jira/FailoverCC_SecondaryIndexes/collectinfo-2018-06-15T131515-ns_1%40172.23.96.89.zip
      https://s3.amazonaws.com/bugdb/jira/FailoverCC_SecondaryIndexes/collectinfo-2018-06-15T131515-ns_1%40172.23.96.91.zip
      https://s3.amazonaws.com/bugdb/jira/FailoverCC_SecondaryIndexes/collectinfo-2018-06-15T131515-ns_1%40172.23.96.92.zip

      Script:

      /opt/jython/bin/jython -J-cp Couchbase-Java-Client-2.5.6/*:jsch-0.1.54.jar:doc_ops.jar testrunner.py -i ${ini_file} -t cbas.cbas_metadata_replication.MetadataReplication.test_failover,cb_bucket_name=default,cbas_bucket_name=default_bucket,cbas_dataset_name=default_ds,items=10000,rebalance_node=CC,nc_nodes_to_add=3,replica_change=0,compression_mode=active,create_secondary_indexes=True,GROUP=G2
      

      Note: It works fine without secondary indexes.

      Attachments

        For Gerrit Dashboard: MB-30135
        # Subject Branch Project Status CR V

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty