Details
-
Bug
-
Resolution: Fixed
-
Critical
-
2.0, 2.2.0
-
Security Level: Public
-
windows 2008 R2 64bit build 2.0.0-1952
Description
Environment:
8 windows server 2008 R2 64 bit in ec2. Each server has 7.5 GB RAM and 45GB EBS disk to store data.
Create a 6 nodes windows cluster in ec2
Loading phase:
- Create one default bucket and load 24 million items to it. Each item has size from 128 to 512 bytes.
Access phase:
- Mutate items with ratio creates/updates/gets/deletes/expirations = 10/60/20/5/5
- Doing rebalance add node, remove node, failover, failover and add back.
- The last test was reboot all 6 nodes in reboot warmup test. During warmup, I see error when collect raw warmup stats
ep_warmup_access_log: corrupt
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.96.107.176:11210 raw warmup
ep_warmup: enabled
ep_warmup_dups: 0
ep_warmup_estimate_time: 20704
ep_warmup_estimated_key_count: 14538479
ep_warmup_estimated_value_count: 14538479
ep_warmup_key_count: 14538479
ep_warmup_keys_time: 745292809
ep_warmup_min_item_threshold: 100
ep_warmup_min_memory_threshold: 100
ep_warmup_oom: 0
ep_warmup_state: done
ep_warmup_thread: complete
ep_warmup_time: 941489106
ep_warmup_value_count: 2285132
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.110.222.56:11210 raw warmup
ep_warmup: enabled
ep_warmup_access_log: corrupt
ep_warmup_dups: 0
ep_warmup_estimate_time: 22194
ep_warmup_estimated_key_count: 23969171
ep_warmup_estimated_value_count: 23969171
ep_warmup_key_count: 23969171
ep_warmup_keys_time: 2466454551
ep_warmup_min_item_threshold: 100
ep_warmup_min_memory_threshold: 100
ep_warmup_oom: 0
ep_warmup_state: done
ep_warmup_thread: complete
ep_warmup_time: 2676578146
ep_warmup_value_count: 976671
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.214.194.95:11210 raw warmup
ep_warmup: enabled
ep_warmup_dups: 0
ep_warmup_estimate_time: 22131
ep_warmup_estimated_key_count: 14540213
ep_warmup_estimated_value_count: 14540213
ep_warmup_key_count: 14540213
ep_warmup_keys_time: 662356019
ep_warmup_min_item_threshold: 100
ep_warmup_min_memory_threshold: 100
ep_warmup_oom: 0
ep_warmup_state: done
ep_warmup_thread: complete
ep_warmup_time: 932353521
ep_warmup_value_count: 2283472
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.46.181.58:11210 raw warmup
Stats 'warmup' are not available from the requested engine.
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.110.218.124:11210 raw warmup
Stats 'warmup' are not available from the requested engine.
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.111.66.215:11210 raw warmup
ep_warmup: enabled
ep_warmup_dups: 0
ep_warmup_estimate_time: 21788
ep_warmup_estimated_key_count: 14679821
ep_warmup_estimated_value_count: 14679821
ep_warmup_key_count: 14679821
ep_warmup_keys_time: 638006011
ep_warmup_min_item_threshold: 100
ep_warmup_min_memory_threshold: 100
ep_warmup_oom: 0
ep_warmup_state: done
ep_warmup_thread: complete
ep_warmup_time: 785227909
ep_warmup_value_count: 2264710
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.2.63.249:11210 raw warmup
ep_warmup: enabled
ep_warmup_access_log: corrupt
ep_warmup_dups: 0
ep_warmup_estimate_time: 26696
ep_warmup_estimated_key_count: 24113568
ep_warmup_estimated_value_count: 24113568
ep_warmup_key_count: 24113568
ep_warmup_keys_time: 1970642317
ep_warmup_min_item_threshold: 100
ep_warmup_min_memory_threshold: 100
ep_warmup_oom: 0
ep_warmup_state: done
ep_warmup_thread: complete
ep_warmup_time: 2221105121
ep_warmup_value_count: 956974
root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.110.206.19:11210 raw warmup
ep_warmup: enabled
ep_warmup_dups: 0
ep_warmup_estimate_time: 20255
ep_warmup_estimated_key_count: 24110614
ep_warmup_estimated_value_count: unknown
ep_warmup_key_count: 21603099
ep_warmup_min_item_threshold: 100
ep_warmup_min_memory_threshold: 100
ep_warmup_oom: 0
ep_warmup_state: loading keys
ep_warmup_thread: running
ep_warmup_value_count: 0
Link to manifest file of this build http://builds.hq.northscale.net/latestbuilds/couchbase-server-enterprise_x86_64_2.0.0-1952-rel.setup.exe.manifest.xml