Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-30498

[FTS System test] Docs missing from Scorch index; upside_down intact

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 6.0.0, 5.5.0
    • Fix Version/s: 6.0.0
    • Component/s: fts

      Description

      Build
      6.0.0-1320

      New System test (not merged yet)
      ./sequoia -scope tests/fts/scope_fts_component.yml -test tests/fts/test_fts_alice_component.yml -version 6.0.0-1320 -provider file:centos_second_cluster.yml

      Steps
      1. In a kv+fts (.206) single node cluster, create 2 buckets - default, other.
      Start loading 10M docs to "other"
      Start loading 10,492,816 to default bucket. Data ops/sec = 10-15k/sec on each bucket
      2. Create scorch and upside_down index, of type default, both on bucket "other".
      3. As data gets indexed, with the workload running in parallel, add 2 nodes- one with kv (.207) and another with kv+fts(.209). Start rebalance.
      4. At the end of rebalance, compare index count of scorch vs upside_down index.

      There are more phases in the system test but the run was aborted here.

      Scorch index : 9996818 @ 99.97%
      Upside down index: 10,000,000 @ 100%

      as seen on UI. No fts activity seen from stats.

      Also attaching screenshot of stats comparing indexes.

      A couple of stat errors:

      1. Both on upside down and scorch, the docs to persist is not 0.
      2. No of docs remaining on scorch index is 0, while indexing progress says 99.97%.

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.5.0-1099 contains cbft commit f9852cf with commit message:
            MB-30498 - Docs missing from index

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.0-1099 contains cbft commit f9852cf with commit message: MB-30498 - Docs missing from index
            Hide
            tai.tran Tai Tran added a comment -

            reopen to add to sprint, will close again after added it. 

            Show
            tai.tran Tai Tran added a comment - reopen to add to sprint, will close again after added it. 
            Hide
            apiravi Aruna Piravi (Inactive) added a comment -

            Sreekanth Sivasankaran, I tested build 6.0.0-1348. Scorch default index on bucket "other" took a very very long time to index but eventually reached 10M (no docs missing from index). Maybe affected by MB-30616?

            For now, I'm comfortable closing this MB because there are no docs missing from index. I will be doing more runs. Will reopen if doc_count does not match.

            Here are logs from the most recent run on 6.0.0-1348 -

            https://s3.amazonaws.com/cb-engineering/Aruna/collectinfo-2018-07-25T184202-ns_1%40172.23.96.207.zip
            https://s3.amazonaws.com/cb-engineering/Aruna/collectinfo-2018-07-25T184202-ns_1%40172.23.96.210.zip

            fts index count on "default" bucket is greater than bucket doc_count by 1. Filing a separate bug.

            Show
            apiravi Aruna Piravi (Inactive) added a comment - Sreekanth Sivasankaran , I tested build 6.0.0-1348. Scorch default index on bucket "other" took a very very long time to index but eventually reached 10M (no docs missing from index). Maybe affected by MB-30616 ? For now, I'm comfortable closing this MB because there are no docs missing from index. I will be doing more runs. Will reopen if doc_count does not match. Here are logs from the most recent run on 6.0.0-1348 - https://s3.amazonaws.com/cb-engineering/Aruna/collectinfo-2018-07-25T184202-ns_1%40172.23.96.207.zip https://s3.amazonaws.com/cb-engineering/Aruna/collectinfo-2018-07-25T184202-ns_1%40172.23.96.210.zip fts index count on "default" bucket is greater than bucket doc_count by 1. Filing a separate bug.
            Hide
            Sreekanth Sivasankaran Sreekanth Sivasankaran added a comment -

            hi Aruna Piravi,  I can see that the fix went in 1354 as per the build tagging above,

            Build couchbase-server-6.0.0-1354 contains cbft commit f9852cf with commit message:
            MB-30498 - Docs missing from index

            So, highly recommend to verify this on a later build..

            thanks,

            Show
            Sreekanth Sivasankaran Sreekanth Sivasankaran added a comment - hi Aruna Piravi ,  I can see that the fix went in 1354 as per the build tagging above, Build couchbase-server-6.0.0-1354 contains cbft commit  f9852cf  with commit message: MB-30498  - Docs missing from index So, highly recommend to verify this on a later build.. thanks,
            Hide
            apiravi Aruna Piravi (Inactive) added a comment -

            Hi Sreekanth Sivasankaran, this issue could not be reproduced on 6.0.0-1375 that contains the fix.. Hence closing as fixed. Thanks!

            Show
            apiravi Aruna Piravi (Inactive) added a comment - Hi Sreekanth Sivasankaran , this issue could not be reproduced on 6.0.0-1375 that contains the fix.. Hence closing as fixed. Thanks!

              People

              • Assignee:
                apiravi Aruna Piravi (Inactive)
                Reporter:
                apiravi Aruna Piravi (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.