Details
-
Bug
-
Resolution: Incomplete
-
1.6.0 beta1
-
None
-
Operating System: CentOS 5.x
Platform: X64
Description
Ran a simple expansion test which does the following
1) start with a single node
2) add a new node to the single and rebalance the cluster
3) remove a the newly added node and rebalance the cluster
All while this was happening, I was running a simple memslap load against the machines. The expansion/contraction test appeared to complete with out issue (see bottom of the report for details), but the output of memslap showed a2b not my_vbucket errors as well as some get misses
servers : 10.253.11.144:11211,10.253.30.47:11211
threads count: 1
concurrency: 16
run time: 600s
windows size: 10k
set proportion: set_prop=0.10
get proportion: get_prop=0.90
<20 SERVER_ERROR a2b not_my_vbucket
...
<7 SERVER_ERROR a2b not_my_vbucket
<7 SERVER_ERROR a2b not_my_vbucket
<7 SERVER_ERROR a2b not_my_vbucket
<7 SERVER_ERROR a2b not_my_vbucket
cmd_get: 157707
cmd_set: 17579
get_misses: 17
verify_misses: 15
verify_failed: 0
written_bytes: 30464285
read_bytes: 175163363
object_bytes: 19125952
[root@domU-12-31-36-00-45-D2 py]# python rebalance_check.py --nodes 10.253.11.144,10.253.30.47 --num_items 1000
validating node 0: 10.253.11.144
validating buckets: PASS
validating items: FEATURE NOT YET IMPLEMENTED
Here is output of the test
Adding 10.253.30.47 to 10.253.11.144: ['python', '/opt/NorthScale/bin/cli/membase', 'rebalance', '--cluster=10.253.11.144:8080', '--server-add=10.253.30.47:8080']
SUCCESS: server added..
rebalance POST response: SUCCESS: rebalance started..
SUCCEEDED
Checking cluster membership...
10.253.11.144: MATCH
10.253.30.47: MATCH
validating buckets: PASS
validating items: FEATURE NOT YET IMPLEMENTED
Removing 10.253.30.47 to 10.253.11.144: ['python', '/opt/NorthScale/bin/cli/membase', 'rebalance', '--cluster=10.253.11.144:8080', '--server-remove=10.253.30.47:8080']
rebalance POST response: SUCCESS: rebalance started..
SUCCEEDED
Checking cluster membership...
10.253.11.144: MATCH
validating buckets: PASS
validating items: FEATURE NOT YET IMPLEMENTED
Logs to be attached