Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42668

Collections: fts multi-collection index reports wrong number of indexed docs.

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Cheshire-Cat
    • None
    • fts

    Description

      Build: 7.0.0-3653

      Test steps are here:

      • create bucket default
      • create scope scope1 in default bucket
      • create 3 collections: collection1, collection2, collection3 in scope1
      • load 10,000 docs into each bucket
      • create the following fts index which uses data from all 3 collections:

       

      {
       "name": "default_index_1",
       "type": "fulltext-index",
       "params": {
        "doc_config": {
         "docid_prefix_delim": "",
         "docid_regexp": "",
         "mode": "scope.collection.type_field",
         "type_field": "type"
        },
        "mapping": {
         "default_analyzer": "standard",
         "default_datetime_parser": "dateTimeOptional",
         "default_field": "_all",
         "default_mapping": {
          "dynamic": false,
          "enabled": false
         },
         "default_type": "_default",
         "docvalues_dynamic": true,
         "index_dynamic": true,
         "store_dynamic": false,
         "type_field": "_type",
         "types": {
          "scope1.collection1": {
           "default_analyzer": "standard",
           "dynamic": true,
           "enabled": true
          },
          "scope1.collection2": {
           "default_analyzer": "standard",
           "dynamic": true,
           "enabled": true
          },
          "scope1.collection3": {
           "default_analyzer": "standard",
           "dynamic": true,
           "enabled": true
          }
         }
        },
        "store": {
         "indexType": "scorch",
         "mossStoreOptions": {},
         "segmentVersion": 15
        }
       },
       "sourceType": "gocbcore",
       "sourceName": "default",
       "sourceUUID": "75de4add7797a4d99491bb6fe6e25e66",
       "sourceParams": {},
       "planParams": {
        "maxPartitionsPerPIndex": 171,
        "numReplicas": 0,
        "indexPartitions": 6
       },
       "uuid": "222c356fd27aca91"
      }
      

      •  update 3,000 docs from collection1 - set expiration time to 30 sec.
      • wait for 30 sec
      • compare docs amount in default bucket: 27,000 and docs, indexed by fts index: 30,000

      fts index reports wrong docs amount both from UI and REST endpoint:

      /api/index/default_index_1/count 

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Same scenario seen when dealing with deletes in the collection. The index doc count doesnt go down in case of multi-collection index when documents are deleted from the collection. This works fine with single collection index.

          mihir.kamdar Mihir Kamdar (Inactive) added a comment - Same scenario seen when dealing with deletes in the collection. The index doc count doesnt go down in case of multi-collection index when documents are deleted from the collection. This works fine with single collection index.

          Thanks, Evgeny Makarenko for capturing this. It looks like a more serious problem with the multi collection indexes. Means, it's not just a problem with the stats alone. 

          Additional verification steps/tests needed for this fix would be,

          1.Create a multi collection index.

          2.Carefully perform some targetted queries on some document's texts.

          3. delete and update those documents targetted by the queries in step 2.

          4. Trigger the queries again to verify that the delete and update operations are reflecting the search hits count as well.

           

          Sreekanth Sivasankaran Sreekanth Sivasankaran added a comment - Thanks, Evgeny Makarenko  for capturing this. It looks like a more serious problem with the multi collection indexes. Means, it's not just a problem with the stats alone.  Additional verification steps/tests needed for this fix would be, 1.Create a multi collection index. 2.Carefully perform some targetted queries on some document's texts. 3. delete and update those documents targetted by the queries in step 2. 4. Trigger the queries again to verify that the delete and update operations are reflecting the search hits count as well.  

          Build couchbase-server-7.0.0-3716 contains cbft commit aff87b9 with commit message:
          MB-42668-FTS multi-collection index reports wrong number of docs

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-3716 contains cbft commit aff87b9 with commit message: MB-42668 -FTS multi-collection index reports wrong number of docs

          People

            evgeny.makarenko Evgeny Makarenko (Inactive)
            evgeny.makarenko Evgeny Makarenko (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty