Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19667

[FTS] term dictionary counts not matching search hit count

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 4.5.1
    • 4.5.0
    • cbft
    • None
    • Untriaged
    • Unknown

    Description

      First observed on performance testing machine.

      Reproduced on my local, with moss enabled, 100k wiki docs loaded:

      $ ~/Documents/research/cbsource/install/bin/cbft-bleve-dump -dictionary revision.text.#text -index wiki_6309b1e5e574a09c_0a44bddb.pindex/ |grep "^called -"
      [FDB INFO] Forestdb blockcache size 134217728 initialized in 2851 us
       
      [FDB INFO] Forestdb opened database file wiki_6309b1e5e574a09c_0a44bddb.pindex//store.0
      called - 399
      

      $ ~/Documents/research/cbsource/install/bin/cbft-bleve-query -field revision.text.#text -queryType term -index wiki_6309b1e5e574a09c_0a44bddb.pindex/ called
      [FDB INFO] Forestdb blockcache size 134217728 initialized in 2316 us
       
      [FDB INFO] Forestdb opened database file wiki_6309b1e5e574a09c_0a44bddb.pindex//store.0
      522 matches, showing 1 through 10, took 21.420104ms
          1. wiki10043700 (0.722119)
          2. wiki10043357 (0.629762)
          3. wiki10081277 (0.600840)
          4. wiki10025405 (0.578984)
          5. wiki10066710 (0.568914)
          6. wiki10042207 (0.568914)
          7. wiki10040428 (0.568914)
          8. wiki10047305 (0.510616)
          9. wiki10081633 (0.507106)
         10. wiki10009765 (0.498090)
       
      [FDB INFO] Forestdb closed database file wiki_6309b1e5e574a09c_0a44bddb.pindex//store.0
      

      399 != 522

      However, the same bucket, when indexed again without moss enabled:

      $ ~/Documents/research/cbsource/install/bin/cbft-bleve-dump -dictionary revision.text.#text -index wiki-nomoss_514a2f0a216a7a59_0a44bddb.pindex/ |grep "^called -"
      [FDB INFO] Forestdb blockcache size 134217728 initialized in 1992 us
       
      [FDB INFO] Forestdb opened database file wiki-nomoss_514a2f0a216a7a59_0a44bddb.pindex//store.0
      called - 522
      

      $ ~/Documents/research/cbsource/install/bin/cbft-bleve-query -field revision.text.#text -queryType term -index wiki-nomoss_514a2f0a216a7a59_0a44bddb.pindex/ called 
      [FDB INFO] Forestdb blockcache size 134217728 initialized in 2045 us
       
      [FDB INFO] Forestdb opened database file wiki-nomoss_514a2f0a216a7a59_0a44bddb.pindex//store.0
      522 matches, showing 1 through 10, took 59.234449ms
          1. wiki10043700 (0.658924)
          2. wiki10043357 (0.574649)
          3. wiki10081277 (0.548258)
          4. wiki10025405 (0.528314)
          5. wiki10066710 (0.519126)
          6. wiki10042207 (0.519125)
          7. wiki10040428 (0.519125)
          8. wiki10047305 (0.465929)
          9. wiki10081633 (0.462727)
         10. wiki10009765 (0.454500)
       
      [FDB INFO] Forestdb closed database file wiki-nomoss_514a2f0a216a7a59_0a44bddb.pindex//store.0
      

      522 == 522

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              steve Steve Yen
              mschoch Marty Schoch [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty