Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46454

[System Test] Deferred index not getting built when moved during rebalance

    XMLWordPrintable

Details

    Description

      Build : 7.0.0-5177 (RC1)
      Test : -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas_scale3.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml
      Scale : 3
      Iteration : 2nd

      There is one index that is not getting built, even when trying to build it manually. The index state remains "Created"

      {
               "bucket" : "bucket7",
               "collection" : "_default",
               "completion" : 0,
               "definition" : "CREATE INDEX `sg_syncDocs_x1` ON `bucket7`((meta().`id`)) WHERE ((meta().`id`) like \"\\\\_sync:%\") WITH {  \"defer_build\":true }",
               "defnId" : 48612999675990305,
               "hosts" : [
                  "172.23.96.253:8091"
               ],
               "indexName" : "sg_syncDocs_x1",
               "indexType" : "plasma",
               "instId" : 17621881332573311286,
               "lastScanTime" : "NA",
               "name" : "sg_syncDocs_x1",
               "numPartition" : 1,
               "numReplica" : 0,
               "partitionMap" : {
                  "172.23.96.253:8091" : [
                     0
                  ]
               },
               "partitioned" : false,
               "progress" : 0,
               "replicaId" : 0,
               "scheduled" : false,
               "scope" : "_default",
               "secExprs" : [
                  "(meta().`id`)"
               ],
               "stale" : false,
               "status" : "Created",
               "where" : "((meta().`id`) like \"\\\\_sync:%\")"
            },
      

      In the indexer logs of 172.23.96.253, I see the following messages :

      indexer.log:2021-05-21T14:12:08.765-07:00 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/48612999675990305
      indexer.log:2021-05-21T14:12:08.765-07:00 [Warn] DDLServiceMgr: There are still index not yet build.  Skip cleaning up build token /indexing/ddl/commandToken/build/48612999675990305.
      indexer.log:2021-05-21T14:22:08.682-07:00 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/48612999675990305
      indexer.log:2021-05-21T14:22:08.682-07:00 [Warn] DDLServiceMgr: There are still index not yet build.  Skip cleaning up build token /indexing/ddl/commandToken/build/48612999675990305.
      

      Similarly, in the query logs, messages like the following can be seen :

      [root@s13001-cnt7 logs]# zgrep -i build query.log*
      query.log:2021-05-21T14:06:25.812-07:00 [Info] BuildIndexes [48612999675990305] ...
      query.log:2021-05-21T14:06:25.832-07:00 [Warn] Index 48612999675990305 has already built. Skip build index.
      query.log:2021-05-21T14:06:25.832-07:00 [Info] BuildIndexes [48612999675990305] - elapsed(19.736414ms), err(<nil>)
      query.log:2021-05-21T14:07:25.446-07:00 [Info] BuildIndexes [48612999675990305] ...
      query.log:2021-05-21T14:07:25.471-07:00 [Warn] Index 48612999675990305 has already built. Skip build index.
      query.log:2021-05-21T14:07:25.471-07:00 [Info] BuildIndexes [48612999675990305] - elapsed(25.011096ms), err(<nil>)
      query.log:2021-05-21T14:13:02.653-07:00 [Info] BuildIndexes [48612999675990305] ...
      query.log:2021-05-21T14:13:02.678-07:00 [Warn] Index 48612999675990305 has already built. Skip build index.
      

      The build index query "build index on bucket7._default._default(sg_syncDocs_x1);" is successful, but doesn't help. I think the only way out here is to drop the index and recreate it.

      Marking this as a regression as this issue hasn't been observed in the longevity test.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            deepkaran.salooja Deepkaran Salooja
            mihir.kamdar Mihir Kamdar (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty