Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60025

[Upgrade] Alternate shard IDs are not generated for the index created in mixed mode cluster state during upgrade from 7.2.1 to 7.6.0

    XMLWordPrintable

Details

    Description

      Steps:

      1. Initialise a 4 node cluster running version 7.2.1-5934
            Node 1 - kv, Node 2 - kv, Node 3 - index-n1ql, Node 4 - index-n1ql
      2. Create a Magma bucket named `bucket-0` with 1 replica, 1024MB RAM per node and 20 collections in it.
      3. Load 5 million documents into the bucket.
      4. Create different types of indexes before starting the upgrade process - primary indexes, secondary indexes with replicas and partitions. Total index count before the upgrade is 62. Index memory quota is set to 5000MB per node.
        The parameter 'redistributeIndexes' is set to True.
      5. Start the upgrade, iterative swap rebalance approach is followed. The 7.6.0 node coming in is installed with a provisioned profile.
      6. After the upgrade of every node, a new collection is created, documents are loaded into it and and an index is built for that new collection (cluster is in mixed mode state).
      7. Index `bucket-0_scope-5_collection-21_index` was created with 1 replica, while in mixed mode state. Index definition was,

      CREATE INDEX `bucket-0_scope-5_collection-21_index` ON `bucket-0`.`scope-5`.`collection-21`(`name` INCLUDE MISSING) WITH { \"num_replica\":1 } 

      The cluster was in this state, 1 index node was upgraded to 7.6.0, while the other index node was still in 7.2.1.

      1. After the complete upgrade of the cluster, new indexes are created.
      2. The parameter 'enableShardAffinity' is set to True.
      3. Swap rebalance of an index node is performed post upgrade. Also, a new index-n1ql node is rebalanced-in after the swap rebalance.
      4. In the end, a validation is done to check if alternate shard IDs are generated for every index, but it is observed that only for the index `bucket-0_scope-5_collection-21_index`, alternate shard ID is not generated for one of the copies of the index. 172.23.216.44 was in 7.6.0 when the index was created.

      {
            "defnId": 1541671972824379000,
            "instId": 4552134876149838300,
            "name": "bucket-0_scope-5_collection-21_index",
            "bucket": "bucket-0",
            "scope": "scope-5",
            "collection": "collection-21",
            "secExprs": [
              "`name`"
            ],
            "indexType": "plasma",
            "status": "Ready",
            "definition": "CREATE INDEX `bucket-0_scope-5_collection-21_index` ON `bucket-0`.`scope-5`.`collection-21`(`name` INCLUDE MISSING) WITH {  \"defer_build\":true, \"nodes\":[ \"172.23.216.44:8091\",\"172.23.216.45:8091\" ], \"num_replica\":1 }",
            "hosts": [
              "172.23.216.45:8091"
            ],
            "completion": 100,
            "progress": 100,
            "scheduled": false,
            "partitioned": false,
            "numPartition": 1,
            "partitionMap": {
              "172.23.216.45:8091": [
                0
              ]
            },
            "numReplica": 1,
            "indexName": "bucket-0_scope-5_collection-21_index",
            "replicaId": 0,
            "stale": false,
            "lastScanTime": "NA",
            "alternateShardIds": {
              "172.23.216.45:8091": {
                "0": [
                  "10399732914857679660-0-0",
                  "10399732914857679660-0-1"
                ]
              }
            }
          },
          {
            "defnId": 1541671972824379000,
            "instId": 17788004084684898000,
            "name": "bucket-0_scope-5_collection-21_index (replica 1)",
            "bucket": "bucket-0",
            "scope": "scope-5",
            "collection": "collection-21",
            "secExprs": [
              "`name`"
            ],
            "indexType": "plasma",
            "status": "Ready",
            "definition": "CREATE INDEX `bucket-0_scope-5_collection-21_index` ON `bucket-0`.`scope-5`.`collection-21`(`name` INCLUDE MISSING) WITH {  \"nodes\":[ \"172.23.216.44:8091\",\"172.23.216.45:8091\" ], \"num_replica\":1 }",
            "hosts": [
              "172.23.216.44:8091"
            ],
            "completion": 100,
            "progress": 100,
            "scheduled": false,
            "partitioned": false,
            "numPartition": 1,
            "partitionMap": {
              "172.23.216.44:8091": [
                0
              ]
            },
            "numReplica": 1,
            "indexName": "bucket-0_scope-5_collection-21_index",
            "replicaId": 1,
            "stale": false,
            "lastScanTime": "Wed Dec  6 22:17:14 PST 2023",
            "alternateShardIds": {}
          } 

      For every other index, alternate shard IDs have been generated for all copies of the index.

      1. Cb-collect logs have been attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            dhananjay.kshirsagar Dhananjay Kshirsagar
            vibhav.sp Vibhav S P
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty