Uploaded image for project: 'Couchbase Lite'
  1. Couchbase Lite
  2. CBL-171

CBL replication never finishes for replication filtering when doc count is 1000

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.6.0
    • 2.7.0
    • LiteCore
    • Security Level: Public
    • CBL2.6.0-126 (net-msft Windows 10)
      SG 2.5.1-11 (Centos 7)
      CBS 6.0.1-2037 (Centos 7)
    • Medium

    Description

      Replication never finishes for test_replication_pull_filtering and test_replication_filter_access_revoke_document when filtering is done with 1000 docs. For 10 docs and 100 docs, test passes.

      The issue is completed doesn't reaches to the exact count of expected total and after checking for sometime (150 iteration) we closes the replicator.

      PS: This is with delta-sync disabled

      pytest --timeout 3600 --liteserv-version=2.6.0-126 --liteserv-host=10.100.161.140 --liteserv-port=8080 --sg-ssl --enable-file-logging --no-conflicts --sync-gateway-version=2.5.1-11 --mode=cc --server-version=6.0.1-2037 --liteserv-platform=net-msft --create-db-per-test=cbl-test testsuites/CBLTester/CBL_Functional_tests/TestSetup_FunctionalTests -k test_replication_pull_filtering[1000]

      Test Report:
      http://uberjenkins.sc.couchbase.com:8080/job/Net-windows-TestSever-Funtional-tests/982/testReport/junit/testsuites.CBLTester.CBL_Functional_tests.TestSetup_FunctionalTests/test_replication_filtering/test_replication_pull_filtering_1000_/

      As one can see because the replication never completes, we got key_error (additional field added in the doc) for the update on doc.

      Attachments

        1. cbcollect_info.zip
          3.44 MB
        2. log_1563779238.80133.rar
          271 kB
        3. logs.txt
          2.53 MB
        4. po.png
          po.png
          3.11 MB
        5. python_client.log
          209 kB
        6. sgcollect_info.zip
          13.50 MB
        7. unit-test.txt
          2 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Jayahari.Vavachan Jay Vavachan added a comment - - edited

          Seems like the status of the replicator is incorrect(i have manually added the progress of the replicator in the lite core, not part of default log).

          CouchbaseLite Replicator Info: {Repl#8} pushStatus=stopped, pullStatus=busy, progress=42792/56992
          

          If we write a unit test with a timeout ~100seconds, and replicate ~400 docs in which 200 should be rejected. then this can be reproduced. Find the attached unit-test.txt and logs.txt

          If i check the db elements count, i can see there are 201 documents. and progress as well.

          cc. Jim Borden

          Jayahari.Vavachan Jay Vavachan added a comment - - edited Seems like the status of the replicator is incorrect(i have manually added the progress of the replicator in the lite core, not part of default log). CouchbaseLite Replicator Info: {Repl#8} pushStatus=stopped, pullStatus=busy, progress=42792/56992 If we write a unit test with a timeout ~100seconds, and replicate ~400 docs in which 200 should be rejected. then this can be reproduced. Find the attached unit-test.txt and logs.txt If i check the db elements count, i can see there are 201 documents. and progress as well. cc. Jim Borden
          Jayahari.Vavachan Jay Vavachan added a comment - - edited

          Assumptions:
          This happens, when the `_activeIncomingRevs < tuning::kMaxActiveIncomingRevs` (100).

          `_waitingChangesMessages` are added with the delayed revisions, which are from the passive to the active. But those changes are not being handled.

          May be calling the `handleMoreChanges`, when rest of the revisions are finished. But not sure on where exactly this happens.

          Jayahari.Vavachan Jay Vavachan added a comment - - edited Assumptions: This happens, when the ` _activeIncomingRevs < tuning::kMaxActiveIncomingRevs` (100). ` _waitingChangesMessages ` are added with the delayed revisions, which are from the passive to the active. But those changes are not being handled. May be calling the ` handleMoreChanges `, when rest of the revisions are finished. But not sure on where exactly this happens.
          Jayahari.Vavachan Jay Vavachan added a comment - Potential fix here: https://github.com/couchbase/couchbase-lite-core/pull/811
          sandy.chuang Sandy Chuang (Inactive) added a comment - https://github.com/couchbase/couchbase-lite-core/pull/812

          Build couchbase-lite-android-2.6.0-149 contains couchbase-lite-core commit 7380a17 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-android-2.6.0-149 contains couchbase-lite-core commit 7380a17 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-core-2.6.0-2193 contains couchbase-lite-core commit 7380a17 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-core-2.6.0-2193 contains couchbase-lite-core commit 7380a17 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-net-2.6.0-127 contains couchbase-lite-core commit 7380a17 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-net-2.6.0-127 contains couchbase-lite-core commit 7380a17 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-ios-2.6.0-141 contains couchbase-lite-core commit 7380a17 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-ios-2.6.0-141 contains couchbase-lite-core commit 7380a17 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Tested on iOS 2.6.0-141. -> both tests passed

          sridevi.saragadam Sridevi Saragadam (Inactive) added a comment - Tested on iOS 2.6.0-141. -> both tests passed
          eunice.huang Eunice Huang (Inactive) added a comment - This issue starts to show in mercury weekly test runs. Please check if the fix has been merged into mercury branch. involved test cases: test_replication_pull_filtering [1000] test_replication_filter_access_revoke_document [1000] jenkins jobs: http://uberjenkins.sc.couchbase.com:8080/job/Android-Listener-TestServer-Functional-tests/1614/testReport/testsuites.CBLTester.CBL_Functional_tests.TestSetup_FunctionalTests/test_replication_filtering/test_replication_pull_filtering_1000_/   http://uberjenkins.sc.couchbase.com:8080/job/Android-Listener-TestServer-Functional-tests/1614/testReport/testsuites.CBLTester.CBL_Functional_tests.TestSetup_FunctionalTests/test_replication_filtering/test_replication_filter_access_revoke_document_1000_/    

          see same issue for iOS functional tests too. Same tests were failing on 2.7.0-27

          sridevi.saragadam Sridevi Saragadam (Inactive) added a comment - see same issue for iOS functional tests too. Same tests were failing on 2.7.0-27
          Jayahari.Vavachan Jay Vavachan added a comment - - edited

          Yea. 7380a17 change is in release/cobalt branch.

          Cherry picked the changes to master PR-838.

          Merged to lite-core master: ef476f590

          Jayahari.Vavachan Jay Vavachan added a comment - - edited Yea. 7380a17 change is in release/cobalt branch. Cherry picked the changes to master PR-838 . Merged to lite-core master: ef476f590

          Build couchbase-lite-android-2.7.0-101 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-android-2.7.0-101 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-net-2.7.0-32 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-net-2.7.0-32 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-core-2.7.0-55 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-core-2.7.0-55 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-log-2.6.0-179 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-log-2.6.0-179 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-cblite-2.6.0-145 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-cblite-2.6.0-145 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-ios-2.7.0-30 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-ios-2.7.0-30 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Tested with build 2.7.0-34 for iOS and 2.7.0-112 for android. Test is passing. .net app is not available for testing, will close the ticket after .net testing.

          hemant.rajput Hemant Rajput added a comment - Tested with build 2.7.0-34 for iOS and 2.7.0-112 for android. Test is passing. .net app is not available for testing, will close the ticket after .net testing.

          Tested with 2.7.0-37 for .net and test is passing. Hence closing the ticket

          hemant.rajput Hemant Rajput added a comment - Tested with 2.7.0-37 for .net and test is passing. Hence closing the ticket

          Build couchbase-lite-cblite-2.6.0-213 contains couchbase-lite-core commit 7380a17 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-cblite-2.6.0-213 contains couchbase-lite-core commit 7380a17 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-log-2.7.0-79 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-log-2.7.0-79 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          Build couchbase-lite-cblite-2.7.0-5 contains couchbase-lite-core commit ef476f5 with commit message:
          CBL-171: Fix Puller not stopping after filter rejects docs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-cblite-2.7.0-5 contains couchbase-lite-core commit ef476f5 with commit message: CBL-171 : Fix Puller not stopping after filter rejects docs

          People

            hemant.rajput Hemant Rajput
            hemant.rajput Hemant Rajput
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty