Details
-
Bug
-
Resolution: Fixed
-
Critical
-
3.4.5
-
None
-
0
Description
Running a query/view/analytics-query workload whilst removing and rebalancing out a node results in consistent errors for that service. It looks like it's continuing to try and do the requests against the node that has been removed. KV works as expected.
Here is a test with a query workload:
http://sdkqe-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-7.0.4-7279-enterprise/Rb1OutQuery-N1QL/04-12-23/058570/614b3d2526ee67e922f32b72b1b2752c-N1QL.html
Starting with 4 nodes:
172.23.111.132:::kv
172.23.111.133:::kv,index,fts,n1ql
172.23.111.134:::kv,index,fts,n1ql
172.23.111.135:::kv,index,fts,n1ql
Then the test removes/rebalances out 172.23.111.133.
And in the logs it is clear that queries are still being sent to that node after it is removed:
2023-04-12 17:21:14.6637||Couchbase.Query.QueryClient|DEBUG|Request canceled Connection refused (172.23.111.133:8093) |
Full set of logs: SdkdConsole.log.zip
There's also similar results for the equivalent analytics test:
http://sdkqe-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-7.0.4-7279/Rb1OutAnalytics-CBAS/04-12-23/043666/d9542e0ac04d0217309081a0a010659c-CBAS.html
And views has some errors:
http://sdkqe-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-7.0.4-7279/Rb2Out-HYBRID/04-12-23/039723/d084516afbbe54ac9986bb4e5245246c-HT.html
Attachments
For Gerrit Dashboard: NCBC-3372 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
189638,12 | NCBC-3372: swap/rebalanced out node continues to be hit with http requests | master | couchbase-net-client | Status: MERGED | +2 | +1 |