Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
7.2.1
-
Untriaged
-
-
0
-
Unknown
Description
Version: couchbase-cloud-server-7.1.5-3830-v1.0.18 |
Test Scenario : Test code
Steps involved in the test:
- 15 scopes and 60 collections across 3 buckets
- Each bucket -> 10M data 1 index on each collection -> 60 indexes
- Sleep - 15 mins
- Run FTS Queries FTS Flex Queries Random fashion for 2 hrs
- Kill CBFT
- sleep for 15 mins
- Scale out to 5 nodes
- Create 10 more fts indexes with following config :
- 1 Index | 1 Replica | 5 Partitions
- 3 Indexes | 2 Replicas | 6 Partitions
- 6 Indexes | 0 Replicas | 4 Partitions
- Sleep for 15 mins
- Run FTS Queries FTS Flex Queries Random fashion for 30 mins again
- Kill CBFT
- Sleep for 15 mins
- Rebalance/Scale in to 4 nodes
Seeing rebalance fail with following reason:
[user:error,2023-07-26T18:01:13.424Z,ns_1@svc-dqs-node-003.ue0ifo-sx0aeba5r.sandbox.nonprod-project-avengers.com:<0.15510.48>:ns_orchestrator:log_rebalance_completion:1433]Rebalance exited with reason {service_rebalance_failed,fts, {agent_died,<34124.20767.32>, {linked_process_died,<34124.5125.45>, {'ns_1@svc-dqs-node-005.ue0ifo-sx0aeba5r.sandbox.nonprod-project-avengers.com', {{badmatch, {false, {topology,[], [<<"a0343ee21ef5a573d140eaa033d64359">>, <<"cef105c7dc4f9ea25f716a94fc6f3262">>, <<"de570d7af6b39240fe0664c43aa17515">>], false, [<<"warning: resource: \"bucket_default1_idx_afrgc-public_likes-country-Overall-free_parking-price\" -- could not meet replication constraints">>, <<"warning: resource: \"bucket_default1_idx_kgzfc-Overall-country-free_parking\" -- could not meet replication constraints">>, <<"warning: resource: \"bucket_default1_idx_onjzu-public_likes-country-price-free_parking-type-Overall\" -- could not meet replication constraints">>, <<"warning: resource: \"bucket_default2_idx_jtgmx-country-public_likes-Overall\" -- could not meet replication constraints">>, <<"warning: resource: \"bucket_default2_idx_wznbg-price-country-free_parking\" -- could not meet replication constraints">>, <<"warning: resource: \"bucket_default3_idx_aptfa-price-public_likes\" -- could not meet replication constraints">>]}, {topology,[], [<<"a0343ee21ef5a573d140eaa033d64359">>, <<"cef105c7dc4f9ea25f716a94fc6f3262">>, <<"de570d7af6b39240fe0664c43aa17515">>], true,[]}}}, [{service_agent,long_poll_worker_loop,5, [{file,"src/service_agent.erl"}, {line,605}]}, {proc_lib,init_p,3, [{file,"proc_lib.erl"},{line,211}]}]}}}}} |