Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
6.5.0
-
Untriaged
-
Yes
Description
build: 6.5.0-4917 , last passed on 6.5.0-4908
- Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql
- Deploy SBM handler with timer
- Start data loading in source bucket
- Add 2 more kv nodes – failed
- Verify results in source bucket
- Remove above added nodes from the cluster
./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers
|
2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'}
|
|
|
Attachments
Issue Links
Activity
Field | Original Value | New Value |
---|---|---|
Assignee | Jeelan Poola [ jeelan.poola ] | Suraj Naik [ suraj.naik ] |
Assignee | Suraj Naik [ suraj.naik ] | Satya Nand [ satya.nand ] |
Component/s | clients [ 10042 ] | |
Component/s | eventing [ 14026 ] |
Labels | functional-test | approved-for-mad-hatter functional-test |
Description |
build: 6.5.0-4917 , last passed on 6.5.0-4908
{noformat} ./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat} 2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
build: 6.5.0-4917 , last passed on 6.5.0-4908
{noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
Description |
build: 6.5.0-4917 , last passed on 6.5.0-4908
{noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * Add 2 more kv nodes * Verify results in source bucket * Remove above added nodes from the cluster {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
Description |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * Add 2 more kv nodes * Verify results in source bucket * Remove above added nodes from the cluster {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * Add 2 more kv nodes * Verify results in source bucket * {color:#de350b}Remove above added nodes from the cluster – failed{color} {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
Description |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * Add 2 more kv nodes * Verify results in source bucket * {color:#de350b}Remove above added nodes from the cluster – failed{color} {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * {color:#de350b}Add 2 more kv nodes – failed{color} * Verify results in source bucket * Remove above added nodes from the cluster – failed {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
Description |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * {color:#de350b}Add 2 more kv nodes – failed{color} * Verify results in source bucket * Remove above added nodes from the cluster – failed {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
build: 6.5.0-4917 , last passed on 6.5.0-4908
* Create 5 node cluster 3 kv , 1 eventing , 1 index and n1ql * Deploy SBM handler with timer * Start data loading in source bucket * {color:#de350b}Add 2 more kv nodes – failed{color} * Verify results in source bucket * Remove above added nodes from the cluster {noformat}./testrunner -i /tmp/testexec.17696.ini -p get-cbcollect-info=True,GROUP=source_bucket_mutation_timers -t eventing.eventing_rebalance.EventingRebalance.test_eventing_rebalance_with_multiple_kv_nodes,doc-per-day=5,dataset=default,nodes_init=5,services_init=kv-kv-kv-eventing-index:n1ql,groups=simple,reset_services=True,handler_code=source_bucket_mutation_with_timers,source_bucket_mutation=True,GROUP=source_bucket_mutation_timers {noformat} {noformat}2019-12-04 21:39:56,138] - [rest_client:3331] ERROR - {u'node': u'ns_1@172.23.104.238', u'code': 0, u'text': u'Rebalance exited with reason {service_rebalance_failed,eventing,\n {worker_died,\n {\'EXIT\',<0.29602.2>,\n {{badmatch,\n {error,\n {unknown_error,\n <<"Some apps are deploying or resuming on some or all Eventing nodes">>}}},\n [{service_rebalancer,rebalance_worker,1,\n [{file,"src/service_rebalancer.erl"},\n {line,170}]},\n {proc_lib,init_p,3,\n [{file,"proc_lib.erl"},{line,232}]}]}}}}.\nRebalance Operation Id = 6868fa7d4f59e481b6b7553fb2b8c450', u'shortText': u'message', u'serverTime': u'2019-12-04T21:39:48.893Z', u'module': u'ns_orchestrator', u'tstamp': 1575524388893, u'type': u'critical'} {noformat} |
Link | This issue blocks MB-36676 [ MB-36676 ] |
Due Date | 12/Dec/19 |
Due Date | 12/Dec/19 | 16/Dec/19 |
Due Date | 16/Dec/19 | 17/Dec/19 |
Resolution | Fixed [ 1 ] | |
Status | Open [ 1 ] | Resolved [ 5 ] |
Status | Resolved [ 5 ] | Closed [ 6 ] |
Link | This issue relates to SDKQE-1914 [ SDKQE-1914 ] |