Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49874

FTS: bleveMaxClauseCount reset to 1024 after restart

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 7.0.0, 6.6.3, 7.1.0, 7.0.2
    • 7.1.0
    • fts
    • Untriaged
    • 1
    • Unknown

    Description

      Build: 7.1.0-1787

      • Create fts cluster
      • Create a bucket and index on it
      • Run a query which hits bleveMaxClauseCount greater than 1024
      • set bleveMaxClauseCount to greater than what above query needs
      • verify the value set with conciseOptions API
      • run the same query - query runs fine
      • restart fts
      • run the same query again - query fails with bleveMaxClauseCount

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Build couchbase-server-7.1.0-1836 contains cbft commit dfb3d3b with commit message:
            MB-49874: Upon process restart, bleve options to be overwritten

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1836 contains cbft commit dfb3d3b with commit message: MB-49874 : Upon process restart, bleve options to be overwritten

            The data update functionality is broken for existing indexes that are opened in the current session. (regression from last fix)

            Steps of reproduction.

            1. Create a beer sample default index and wait for indexing to complete.
            2. Shutdown cluster run.
            3. Start the cluster again.
            4. Goto bucket and edit some text and then search the new text in the index.
            5. Confirm whether you get those new updates in the search results.

            Root cause => the original fix here made the analysis workers to reset at this point - https://github.com/couchbase/cbft/blob/5cb62de5ab305604a141258c3dbb3b7840629388/cmd/cbft/init_bleve.go#L88 

            But by the time this reset happens, many of the existing scorch indexes would have already opened with the older workers from here(https://github.com/couchbase/cbft/blob/3fc4af32e1426d8ce07af03c44cb72b3f461b509/cmd/cbft/main.go#L376) which are in a closed state now due to the reset that happened above.

            So, no further indexing/mutations would be handled in those opened/old indexes since their analysis queue are closed. And they never get linked to the new refreshed/reset analysis queue.

             

            Girish Benakappa, It is surprising that this regression got escaped in the automated weekly tests.

            We need to cover such cases in our automated tests so that these are never missed in the future.

             

             

            Sreekanth Sivasankaran Sreekanth Sivasankaran added a comment - The data update functionality is broken for existing indexes that are opened in the current session. (regression from last fix) Steps of reproduction. Create a beer sample default index and wait for indexing to complete. Shutdown cluster run. Start the cluster again. Goto bucket and edit some text and then search the new text in the index. Confirm whether you get those new updates in the search results. Root cause => the original fix here made the analysis workers to reset at this point - https://github.com/couchbase/cbft/blob/5cb62de5ab305604a141258c3dbb3b7840629388/cmd/cbft/init_bleve.go#L88   But by the time this reset happens, many of the existing scorch indexes would have already opened with the older workers from here( https://github.com/couchbase/cbft/blob/3fc4af32e1426d8ce07af03c44cb72b3f461b509/cmd/cbft/main.go#L376 ) which are in a closed state now due to the reset that happened above. So, no further indexing/mutations would be handled in those opened/old indexes since their analysis queue are closed. And they never get linked to the new refreshed/reset analysis queue.   Girish Benakappa , It is surprising that this regression got escaped in the automated weekly tests. We need to cover such cases in our automated tests so that these are never missed in the future.    

            Sreekanth Sivasankaran
            We do have a case where we mutate docs and check if mutated documents are processed. But we don't have a case where we restart couchbase before update.

            It seems this is reproducible only when we restart couchbase before update but not consistently (still checking whats. missing). Will add this flow to our suite.

            girish.benakappa Girish Benakappa added a comment - Sreekanth Sivasankaran We do have a case where we mutate docs and check if mutated documents are processed. But we don't have a case where we restart couchbase before update. It seems this is reproducible only when we restart couchbase before update but not consistently (still checking whats. missing). Will add this flow to our suite.

            Build couchbase-server-7.1.0-2149 contains cbft commit df0d489 with commit message:
            MB-49874 - bleveMaxClauseCount reset to 1024 after restart

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-2149 contains cbft commit df0d489 with commit message: MB-49874 - bleveMaxClauseCount reset to 1024 after restart

            Build couchbase-server-7.1.0-2149 contains cbgt commit d30a0f4 with commit message:
            MB-49874 - bleveMaxClauseCount reset to 1024 after restart

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-2149 contains cbgt commit d30a0f4 with commit message: MB-49874 - bleveMaxClauseCount reset to 1024 after restart

            Verified with 7.1.0-2202

            girish.benakappa Girish Benakappa added a comment - Verified with 7.1.0-2202

            People

              girish.benakappa Girish Benakappa
              girish.benakappa Girish Benakappa
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty