Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.5.1, 5.0.0
-
Untriaged
-
Yes
Description
Build : 4.7.0-1021
Test : ./testrunner -i INI_FILE.ini get-cbcollect-info=True,get-coredumps=True,get-logs=False,stop-on-failure=False,GROUP=P0 -t fts.moving_topology_fts.MovingTopFTS.graceful_failover_and_delta_recovery_during_index_building,items=30000,cluster=D,D+F,GROUP=P0
This issue wasnt seen the in previous or newer builds, so this would be an intermittent issue. However, if reproduced again, I will update the bug with details.
Steps:
1. D(.96),F(.190) cluster
2. Begin index building on F with 30K keys.
3. In parallel, trigger graceful failover for F
4. After successful failover, add the node back to the cluster with recoveryType=delta.
4. Rebalance completes successfully.
5. Index building also completes successfully.
However, a crash is seen in the memcached log at 2016-08-18T17:24:47.046418-07:00, which is post the successful test completion, but when the cleanup is in progress.
[2016-08-18 17:24:45,781] - [remote_util:206] INFO - Connected to 172.23.105.190
|
[2016-08-18 17:24:46,350] - [remote_util:2383] INFO - running command.raw on 172.23.105.190: ls /opt/couchbase/var/lib/couchbase/data/@fts |grep default_index_1*.pindex | wc -l
|
[2016-08-18 17:24:46,444] - [remote_util:2420] INFO - command executed successfully
|
[2016-08-18 17:24:46,444] - [fts_base:886] INFO - 0
|
[2016-08-18 17:24:46,446] - [fts_base:670] INFO - Validated: all index files for default_index_1 deleted from disk
|
[2016-08-18 17:24:46,446] - [fts_base:978] INFO - removing nodes from cluster ...
|
[2016-08-18 17:24:46,461] - [fts_base:980] INFO - cleanup [ip:172.23.105.96 port:8091 ssh_username:root, ip:172.23.105.190 port:8091 ssh_username:root]
|
[2016-08-18 17:24:46,488] - [bucket_helper:138] INFO - deleting existing buckets [u'default'] on 172.23.105.96
|
[2016-08-18 17:24:46,489] - [bucket_helper:140] INFO - remove bucket default ...
|
[2016-08-18 17:24:48,378] - [bucket_helper:154] INFO - deleted bucket : default from 172.23.105.96
|
[2016-08-18 17:24:48,378] - [bucket_helper:230] INFO - waiting for bucket deletion to complete....
|
