Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.5.0
-
Untriaged
-
-
Unknown
Description
Build
4.5.0-2151
Testcase
./testrunner -i INI_FILE.ini -p skip-cleanup=True,get-cbcollect-info=True,get-logs=False,stop-on-failure=False,GROUP=P1 -t fts.moving_topology_fts.MovingTopFTS.rebalance_out_during_index_building,items=30000,cluster=D,F,F,index_replicas=1,standard_buckets=2,sasl_buckets=2,GROUP=P1
Same testcase whose failure led to MB-19117 and sometimes rebalance failure due to conflicting metakv changes (MB-19037). It's now MB-19037 followed by OOM causing rebalance failure.
2016-04-19 17:35:47 | INFO | MainProcess | test_thread | [moving_topology_fts.rebalance_out_during_index_building] Index building has begun...
|
2016-04-19 17:35:49 | INFO | MainProcess | test_thread | [moving_topology_fts.rebalance_out_during_index_building] Index count for default_index_1: 5925
|
2016-04-19 17:35:52 | INFO | MainProcess | test_thread | [moving_topology_fts.rebalance_out_during_index_building] Index count for sasl_bucket_1_index_1: 5091
|
2016-04-19 17:35:54 | INFO | MainProcess | test_thread | [moving_topology_fts.rebalance_out_during_index_building] Index count for sasl_bucket_2_index_1: 2457
|
2016-04-19 17:35:56 | INFO | MainProcess | test_thread | [moving_topology_fts.rebalance_out_during_index_building] Index count for standard_bucket_1_index_1: 2084
|
2016-04-19 17:35:58 | INFO | MainProcess | test_thread | [moving_topology_fts.rebalance_out_during_index_building] Index count for standard_bucket_2_index_1: 513
|
2016-04-19 17:35:58 | INFO | MainProcess | test_thread | [fts_base.__async_rebalance_out] Starting rebalance-out nodes:[ip:172.23.106.176 port:8091 ssh_username:root] at C1 cluster 172.23.106.139
|
2016-04-19 17:35:58 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance params : password=password&ejectedNodes=ns_1%40172.23.106.176&user=Administrator&knownNodes=ns_1%40172.23.106.176%2Cns_1%40172.23.106.175%2Cns_1%40172.23.106.139
|
2016-04-19 17:35:59 | INFO | MainProcess | Cluster_Thread | [rest_client.rebalance] rebalance operation started
|
2016-04-19 17:35:59 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 0.00 %
|
2016-04-19 17:36:09 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 17.99 %
|
2016-04-19 17:36:19 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 18.63 %
|
2016-04-19 17:36:29 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 18.63 %
|
2016-04-19 17:36:39 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:36:49 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:36:59 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:37:09 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:37:19 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:37:29 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:37:39 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:37:49 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:37:59 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:38:09 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:38:20 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:38:30 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:38:40 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:38:50 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:39:00 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:39:10 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:39:20 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:39:30 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:39:40 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:39:50 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
2016-04-19 17:40:00 | INFO | MainProcess | Cluster_Thread | [rest_client._rebalance_progress] rebalance percentage : 19.61 %
|
There are OOMs on .175.
