Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37340

index scan timeout in RC4 build Intermittently during syncgateway load testing

    XMLWordPrintable

Details

    • Untriaged
    • Unknown

    Description

      Observing  intermittent indexer failures in syncgateway load tests which is eventually resulting some query failures on syncgateway side .

      cb_collect_info logs :

      [^collectinfo-2019-12-23T214220-ns_1@172.23.100.190.zip]

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Hi Ajay Bhullar

            This bug fix reverses the fix done for MB-32087. For current fix, it would be good to cover below scenarios:
            1. Regression tests covering all three consistency options (Any, Request Plus and Query consistency).
            2. Without any rollback or bucket flush: It would be good to run repeated scans with all three consistency options and fast snapshots generation by indexer - for example:
            a. MOI storage with simple index on single small field
            b. MOI array index on an array that does not exist in the document and MOI storage. Ex: CREATE INDEX `idx1` ON `bucket-1`(all array1)
            And, array1 is missing in the document; since array1 is missing, there is not much work done on storage ensuring fast snapshots.
            3. Bucket flush: A request_plus scan right after a bucket flush will return results instead of 0. Please note, this point is not a validation of the fix but really just an old behavior we are reverting to. The tracking bug for this issue is MB-32087.

            Sharath Sulochana: Please update when the Sync Gateway load tests have run successfully for multiple iterations with the fix.

            prathibha Prathibha Bisarahalli (Inactive) added a comment - - edited Hi Ajay Bhullar This bug fix reverses the fix done for MB-32087. For current fix, it would be good to cover below scenarios: 1. Regression tests covering all three consistency options (Any, Request Plus and Query consistency). 2. Without any rollback or bucket flush: It would be good to run repeated scans with all three consistency options and fast snapshots generation by indexer - for example: a. MOI storage with simple index on single small field b. MOI array index on an array that does not exist in the document and MOI storage. Ex: CREATE INDEX `idx1` ON `bucket-1`(all array1) And, array1 is missing in the document; since array1 is missing, there is not much work done on storage ensuring fast snapshots. 3. Bucket flush: A request_plus scan right after a bucket flush will return results instead of 0. Please note, this point is not a validation of the fix but really just an old behavior we are reverting to. The tracking bug for this issue is MB-32087. Sharath Sulochana : Please update when the Sync Gateway load tests have run successfully for multiple iterations with the fix.

            Prathibha Bisarahalli -All Sync Gateway tests are successful.  No more intermittent timeout issues .

            sharath.sulochana Sharath Sulochana (Inactive) added a comment - Prathibha Bisarahalli  -All Sync Gateway tests are successful.  No more intermittent timeout issues .
            ajay.bhullar Ajay Bhullar added a comment -

            all the above test cases have been tested, all queries were run ~2 hours for each scan_consistency, they always returned the same results each time, and the amount of time each query took was similar (for 2 a and b). For the third test case, girish has confirmed we see the behavior detailed in MB-32087 before the fix went in. Regression looks clean

            ajay.bhullar Ajay Bhullar added a comment - all the above test cases have been tested, all queries were run ~2 hours for each scan_consistency, they always returned the same results each time, and the amount of time each query took was similar (for 2 a and b). For the third test case, girish has confirmed we see the behavior detailed in MB-32087 before the fix went in. Regression looks clean
            ajay.bhullar Ajay Bhullar added a comment -

            closing as all the testing detailed above went well and SG tests are no longer intermittently failing

            ajay.bhullar Ajay Bhullar added a comment - closing as all the testing detailed above went well and SG tests are no longer intermittently failing

            Build couchbase-server-7.0.0-1194 contains indexing commit f631c36 with commit message:
            MB-37340: Disable session_consistency_strict scans

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-1194 contains indexing commit f631c36 with commit message: MB-37340 : Disable session_consistency_strict scans

            People

              deepkaran.salooja Deepkaran Salooja
              sharath.sulochana Sharath Sulochana (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty