Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7230

[system test] warmup access log corrupt during warmup in windows

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 2.5.0
    • 2.0, 2.2.0
    • couchbase-bucket
    • Security Level: Public
    • windows 2008 R2 64bit build 2.0.0-1952

    Description

      Environment:
      8 windows server 2008 R2 64 bit in ec2. Each server has 7.5 GB RAM and 45GB EBS disk to store data.
      Create a 6 nodes windows cluster in ec2

      Loading phase:

      • Create one default bucket and load 24 million items to it. Each item has size from 128 to 512 bytes.

      Access phase:

      • Mutate items with ratio creates/updates/gets/deletes/expirations = 10/60/20/5/5
      • Doing rebalance add node, remove node, failover, failover and add back.
      • The last test was reboot all 6 nodes in reboot warmup test. During warmup, I see error when collect raw warmup stats
        ep_warmup_access_log: corrupt

      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.96.107.176:11210 raw warmup
      ep_warmup: enabled
      ep_warmup_dups: 0
      ep_warmup_estimate_time: 20704
      ep_warmup_estimated_key_count: 14538479
      ep_warmup_estimated_value_count: 14538479
      ep_warmup_key_count: 14538479
      ep_warmup_keys_time: 745292809
      ep_warmup_min_item_threshold: 100
      ep_warmup_min_memory_threshold: 100
      ep_warmup_oom: 0
      ep_warmup_state: done
      ep_warmup_thread: complete
      ep_warmup_time: 941489106
      ep_warmup_value_count: 2285132
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.110.222.56:11210 raw warmup
      ep_warmup: enabled
      ep_warmup_access_log: corrupt
      ep_warmup_dups: 0
      ep_warmup_estimate_time: 22194
      ep_warmup_estimated_key_count: 23969171
      ep_warmup_estimated_value_count: 23969171
      ep_warmup_key_count: 23969171
      ep_warmup_keys_time: 2466454551
      ep_warmup_min_item_threshold: 100
      ep_warmup_min_memory_threshold: 100
      ep_warmup_oom: 0
      ep_warmup_state: done
      ep_warmup_thread: complete
      ep_warmup_time: 2676578146
      ep_warmup_value_count: 976671
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.214.194.95:11210 raw warmup
      ep_warmup: enabled
      ep_warmup_dups: 0
      ep_warmup_estimate_time: 22131
      ep_warmup_estimated_key_count: 14540213
      ep_warmup_estimated_value_count: 14540213
      ep_warmup_key_count: 14540213
      ep_warmup_keys_time: 662356019
      ep_warmup_min_item_threshold: 100
      ep_warmup_min_memory_threshold: 100
      ep_warmup_oom: 0
      ep_warmup_state: done
      ep_warmup_thread: complete
      ep_warmup_time: 932353521
      ep_warmup_value_count: 2283472
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.46.181.58:11210 raw warmup
      Stats 'warmup' are not available from the requested engine.
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.110.218.124:11210 raw warmup
      Stats 'warmup' are not available from the requested engine.
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.111.66.215:11210 raw warmup
      ep_warmup: enabled
      ep_warmup_dups: 0
      ep_warmup_estimate_time: 21788
      ep_warmup_estimated_key_count: 14679821
      ep_warmup_estimated_value_count: 14679821
      ep_warmup_key_count: 14679821
      ep_warmup_keys_time: 638006011
      ep_warmup_min_item_threshold: 100
      ep_warmup_min_memory_threshold: 100
      ep_warmup_oom: 0
      ep_warmup_state: done
      ep_warmup_thread: complete
      ep_warmup_time: 785227909
      ep_warmup_value_count: 2264710
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.2.63.249:11210 raw warmup
      ep_warmup: enabled
      ep_warmup_access_log: corrupt
      ep_warmup_dups: 0
      ep_warmup_estimate_time: 26696
      ep_warmup_estimated_key_count: 24113568
      ep_warmup_estimated_value_count: 24113568
      ep_warmup_key_count: 24113568
      ep_warmup_keys_time: 1970642317
      ep_warmup_min_item_threshold: 100
      ep_warmup_min_memory_threshold: 100
      ep_warmup_oom: 0
      ep_warmup_state: done
      ep_warmup_thread: complete
      ep_warmup_time: 2221105121
      ep_warmup_value_count: 956974
      root@ip-10-116-233-143:~/testrunner# /opt/couchbase/bin/cbstats 10.110.206.19:11210 raw warmup
      ep_warmup: enabled
      ep_warmup_dups: 0
      ep_warmup_estimate_time: 20255
      ep_warmup_estimated_key_count: 24110614
      ep_warmup_estimated_value_count: unknown
      ep_warmup_key_count: 21603099
      ep_warmup_min_item_threshold: 100
      ep_warmup_min_memory_threshold: 100
      ep_warmup_oom: 0
      ep_warmup_state: loading keys
      ep_warmup_thread: running
      ep_warmup_value_count: 0

      Link to manifest file of this build http://builds.hq.northscale.net/latestbuilds/couchbase-server-enterprise_x86_64_2.0.0-1952-rel.setup.exe.manifest.xml

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            thuan Thuan Nguyen
            thuan Thuan Nguyen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty