Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
2.6.0
-
Security Level: Public
-
CBL2.6.0-126 (net-msft Windows 10)
SG 2.5.1-11 (Centos 7)
CBS 6.0.1-2037 (Centos 7)
-
Medium
Description
Replication never finishes for test_replication_pull_filtering and test_replication_filter_access_revoke_document when filtering is done with 1000 docs. For 10 docs and 100 docs, test passes.
The issue is completed doesn't reaches to the exact count of expected total and after checking for sometime (150 iteration) we closes the replicator.
PS: This is with delta-sync disabled
pytest --timeout 3600 --liteserv-version=2.6.0-126 --liteserv-host=10.100.161.140 --liteserv-port=8080 --sg-ssl --enable-file-logging --no-conflicts --sync-gateway-version=2.5.1-11 --mode=cc --server-version=6.0.1-2037 --liteserv-platform=net-msft --create-db-per-test=cbl-test testsuites/CBLTester/CBL_Functional_tests/TestSetup_FunctionalTests -k test_replication_pull_filtering[1000]
As one can see because the replication never completes, we got key_error (additional field added in the doc) for the update on doc.
Attachments
Activity
| Field | Original Value | New Value |
|---|---|---|
| Attachment | log_1563779238.80133.rar [ 71186 ] |
| Summary | CBL replication never finishes for pull filtering when doc count is 1000 | CBL replication never finishes for replication filtering when doc count is 1000 |
| Description |
Replication never finishes for test_replication_pull_filtering when pull filtering is done with 1000 docs. For 10 docs and 100 docs, test passes.
The issue is completed doesn't reaches to the exact count of expected total and after checking for sometime (150 iteration) we closes the replicator. PS: This is with delta-sync disabled pytest --timeout 3600 --liteserv-version=2.6.0-126 --liteserv-host=10.100.161.140 --liteserv-port=8080 --sg-ssl --enable-file-logging --no-conflicts --sync-gateway-version=2.5.1-11 --mode=cc --server-version=6.0.1-2037 --liteserv-platform=net-msft --create-db-per-test=cbl-test testsuites/CBLTester/CBL_Functional_tests/TestSetup_FunctionalTests -k test_replication_pull_filtering[1000] Test Report: http://uberjenkins.sc.couchbase.com:8080/job/Net-windows-TestSever-Funtional-tests/982/testReport/junit/testsuites.CBLTester.CBL_Functional_tests.TestSetup_FunctionalTests/test_replication_filtering/test_replication_pull_filtering_1000_/ As one can see because the replication never completes, we got key_error (additional field added in the doc) for the update on doc. |
Replication never finishes for test_replication_pull_filtering and test_replication_filter_access_revoke_document when filtering is done with 1000 docs. For 10 docs and 100 docs, test passes.
The issue is completed doesn't reaches to the exact count of expected total and after checking for sometime (150 iteration) we closes the replicator. PS: This is with delta-sync disabled pytest --timeout 3600 --liteserv-version=2.6.0-126 --liteserv-host=10.100.161.140 --liteserv-port=8080 --sg-ssl --enable-file-logging --no-conflicts --sync-gateway-version=2.5.1-11 --mode=cc --server-version=6.0.1-2037 --liteserv-platform=net-msft --create-db-per-test=cbl-test testsuites/CBLTester/CBL_Functional_tests/TestSetup_FunctionalTests -k test_replication_pull_filtering[1000] Test Report: http://uberjenkins.sc.couchbase.com:8080/job/Net-windows-TestSever-Funtional-tests/982/testReport/junit/testsuites.CBLTester.CBL_Functional_tests.TestSetup_FunctionalTests/test_replication_filtering/test_replication_pull_filtering_1000_/ As one can see because the replication never completes, we got key_error (additional field added in the doc) for the update on doc. |
| Assignee | The Lite [ cbgto ] | Sandy Chuang [ sandy.chaung ] |
| Component/s | iOS [ 15348 ] |
| Attachment | unit-test.txt [ 71268 ] |
| Attachment | logs.txt [ 71269 ] |
| Attachment | po.png [ 71270 ] |
Assumptions:
This happens, when the `_activeIncomingRevs < tuning::kMaxActiveIncomingRevs` (100).
`_waitingChangesMessages` are added with the delayed revisions, which are from the passive to the active. But those changes are not being handled.
May be calling the `handleMoreChanges`, when rest of the revisions are finished. But not sure on where exactly this happens.
Potential fix here: https://github.com/couchbase/couchbase-lite-core/pull/811
| Fix Version/s | Cobalt [ 16173 ] |
| Component/s | LiteCore [ 15347 ] | |
| Component/s | .NET [ 15346 ] | |
| Component/s | iOS [ 15348 ] |
| Assignee | Sandy Chuang [ sandy.chaung ] | Jens Alfke [ jens ] |
Build couchbase-lite-android-2.6.0-149 contains couchbase-lite-core commit 7380a17 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-core-2.6.0-2193 contains couchbase-lite-core commit 7380a17 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Automated transition triggered when Jens Alfke merged pull request #812 in GitHub -
| Resolution | Fixed [ 1 ] | |
| Status | Open [ 1 ] | Resolved [ 5 ] |
| Actual End | 2019-07-30 17:09 (issue has been closed) |
Build couchbase-lite-net-2.6.0-127 contains couchbase-lite-core commit 7380a17 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-ios-2.6.0-141 contains couchbase-lite-core commit 7380a17 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
| Remote Link | This issue links to "Page (Couchbase, Inc. Wiki)" [ 18339 ] |
| Assignee | Jens Alfke [ jens ] | Hemant Rajput [ hemant.rajput ] |
| Remote Link | This issue links to "Page (Couchbase, Inc. Wiki)" [ 18339 ] |
This issue starts to show in mercury weekly test runs. Please check if the fix has been merged into mercury branch.
involved test cases:
test_replication_pull_filtering[1000]
test_replication_filter_access_revoke_document[1000]
jenkins jobs:
Â
Â
Â
| Assignee | Hemant Rajput [ hemant.rajput ] | The Lite [ cbgto ] |
| Resolution | Fixed [ 1 ] | |
| Status | Resolved [ 5 ] | Reopened [ 4 ] |
see same issue for iOS functional tests too. Same tests were failing on 2.7.0-27
| Fix Version/s | Mercury [ 16188 ] | |
| Fix Version/s | 2.6.0 [ 16173 ] |
Yea. 7380a17 change is in release/cobalt branch.
Cherry picked the changes to master PR-838.
Merged to lite-core master: ef476f590
Build couchbase-lite-android-2.7.0-101 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Automated transition triggered when Jens Alfke merged pull request #838 in GitHub -
| Resolution | Fixed [ 1 ] | |
| Status | Reopened [ 4 ] | Resolved [ 5 ] |
Build couchbase-lite-net-2.7.0-32 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-core-2.7.0-55 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-log-2.6.0-179 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-cblite-2.6.0-145 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-ios-2.7.0-30 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Tested with build 2.7.0-34 for iOS and 2.7.0-112 for android. Test is passing. .net app is not available for testing, will close the ticket after .net testing.
| Assignee | The Lite [ cbgto ] | Hemant Rajput [ hemant.rajput ] |
Tested with 2.7.0-37 for .net and test is passing. Hence closing the ticket
| Status | Resolved [ 5 ] | Closed [ 6 ] |
Build couchbase-lite-cblite-2.6.0-213 contains couchbase-lite-core commit 7380a17 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-log-2.7.0-79 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Build couchbase-lite-cblite-2.7.0-5 contains couchbase-lite-core commit ef476f5 with commit message:
CBL-171: Fix Puller not stopping after filter rejects docs
Seems like the status of the replicator is incorrect(i have manually added the progress of the replicator in the lite core, not part of default log).
CouchbaseLite Replicator Info: {Repl#8} pushStatus=stopped, pullStatus=busy, progress=42792/56992If we write a unit test with a timeout ~100seconds, and replicate ~400 docs in which 200 should be rejected. then this can be reproduced. Find the attached unit-test.txt
and logs.txt
If i check the db elements count, i can see there are 201 documents. and progress as well.
cc. Jim Borden