Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
7.6.0
-
Operating System : Suse Linux 12
Couchbase Enterprise Edition build 7.6.0-2090
-
Untriaged
-
Linux x86_64
-
-
0
-
Unknown
Description
Steps to reproduce
- Created a 1 node kv cluster
- Created a couchstore bucket named "default"
- Three views default_view0, default_view1, default_view2 was created successfully in ddoc: ddoc1
- A lot of view queries were running continuously throughout the test
- Added 2 more nodes and started a rebalance - Rebalance succeeds
- Added one more node and started rebalance - Rebalance fails
2024-02-04T13:09:48.148Z, ns_vbucket_mover:0:critical:message(ns_1@ec2-3-86-84-232.compute-1.amazonaws.com) - Worker <0.19697.9> (for action {move,{817, ['ns_1@ec2-34-239-101-114.compute-1.amazonaws.com', 'ns_1@ec2-3-86-84-232.compute-1.amazonaws.com'], ['ns_1@ec2-18-212-13-207.compute-1.amazonaws.com', 'ns_1@ec2-3-86-84-232.compute-1.amazonaws.com'], []}}) exited with reason {unexpected_exit, {'EXIT', <0.20719.9>, {{bulk_set_vbucket_state_failed, [{'ns_1@ec2-3-86-84-232.compute-1.amazonaws.com', {'EXIT', {{error, {badrpc, {'EXIT', {{badarg, [{ets, lookup, [set_view_by_ddoc_id_ets, {<<"default">>, <<"_design/ddoc1">>}], [{error_info, #{cause => id, module => erl_stdlib_errors}}]}, {couch_set_view_ddoc_cache, get_ddoc, 2, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view_ddoc_cache.erl"}, {line, 50}]}, {couch_set_view_group, open_set_group, 3, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view_group.erl"}, {line, 1779}]}, {couch_set_view, open_set_group, 4, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view.erl"}, {line, 445}]}, {couch_set_view, get_group_pid, 4, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view.erl"}, {line, 132}]}, {couch_set_view, trigger_update, 4, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view.erl"}, {line, 374}]}, {capi_set_view_manager, '-handle_call/3-lc$^1/1-0-', 2, [{file, "src/capi_set_view_manager.erl"}, {line, 218}]}, {capi_set_view_manager, handle_call, 3, [{file, "src/capi_set_view_manager.erl"}, {line, 220}]}]}, {gen_server, call, ['capi_set_view_manager-default', initiate_indexing, infinity]}}}}}, {gen_server, call, [{'janitor_agent-default', 'ns_1@ec2-3-86-84-232.compute-1.amazonaws.com'}, {if_rebalance, <0.21844.8>, {update_vbucket_state, 817, replica, undefined, 'ns_1@ec2-18-212-13-207.compute-1.amazonaws.com'}}, infinity]}}}}]}, [{janitor_agent, bulk_set_vbucket_state, 4, [{file, "src/janitor_agent.erl"}, {line, 404}]}, {ns_single_vbucket_mover, update_replication_post_move, 5, [{file, "src/ns_single_vbucket_mover.erl"}, {line, 518}]}, {ns_single_vbucket_mover, on_move_done_body, 6, [{file, "src/ns_single_vbucket_mover.erl"}, {line, 538}]}, {proc_lib, init_p,3, [{file, "proc_lib.erl"}, {line, 225}]}]}}}2024-02-04T13:09:48.214Z, ns_orchestrator:0:critical:message(ns_1@ec2-3-86-84-232.compute-1.amazonaws.com) - Rebalance exited with reason {mover_crashed, {unexpected_exit, {'EXIT',<0.20719.9>, {{bulk_set_vbucket_state_failed, [{'ns_1@ec2-3-86-84-232.compute-1.amazonaws.com', {'EXIT', {{error, {badrpc, {'EXIT', {{badarg, [{ets,lookup, [set_view_by_ddoc_id_ets, {<<"default">>, <<"_design/ddoc1">>}], [{error_info, #{cause => id, module => erl_stdlib_errors}}]}, {couch_set_view_ddoc_cache, get_ddoc,2, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view_ddoc_cache.erl"}, {line,50}]}, {couch_set_view_group, open_set_group,3, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view_group.erl"}, {line,1779}]}, {couch_set_view,open_set_group,4, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view.erl"}, {line,445}]}, {couch_set_view,get_group_pid,4, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view.erl"}, {line,132}]}, {couch_set_view,trigger_update,4, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_set_view/src/couch_set_view.erl"}, {line,374}]}, {capi_set_view_manager, '-handle_call/3-lc$^1/1-0-',2, [{file, "src/capi_set_view_manager.erl"}, {line,218}]}, {capi_set_view_manager, handle_call,3, [{file, "src/capi_set_view_manager.erl"}, {line,220}]}]}, {gen_server,call, ['capi_set_view_manager-default', initiate_indexing,infinity]}}}}}, {gen_server,call, [{'janitor_agent-default', 'ns_1@ec2-3-86-84-232.compute-1.amazonaws.com'}, {if_rebalance,<0.21844.8>, {update_vbucket_state,817,replica, undefined, 'ns_1@ec2-18-212-13-207.compute-1.amazonaws.com'}}, infinity]}}}}]}, [{janitor_agent,bulk_set_vbucket_state,4, [{file,"src/janitor_agent.erl"}, {line,404}]}, {ns_single_vbucket_mover, update_replication_post_move,5, [{file,"src/ns_single_vbucket_mover.erl"}, {line,518}]}, {ns_single_vbucket_mover,on_move_done_body, 6, [{file,"src/ns_single_vbucket_mover.erl"}, {line,538}]}, {proc_lib,init_p,3, [{file,"proc_lib.erl"},{line,225}]}]}}}}.Rebalance Operation Id = 2c386912763139f020cb136b6cb85061 |
Testrunner script to reproduce
./testrunner -i /data/workspace/suse12-p0-os_certify-vset00-00-view/testexec.1580.ini -p get-delays=true,get-cbcollect-info=False,get-cbcollect-info=True,get-cbcollect-info=True,get-cbcollect-info=True,hostname=true,sirius_url=http://172.23.120.103:4000 -t rebalance.rebalancein.RebalanceInTests.incremental_rebalance_in_with_queries,bucket_storage=couchstore,blob_generator=False,items=2000,is_dev_ddoc=False,max_verify=2000,GROUP=P1 |
Job name : suse12-os-certify-view
Attachments
Issue Links
- relates to
-
-
- Closed
-