Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5806

stats_collector:handle_info:96] Exception in stats collector

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Critical
    • 2.0-beta
    • None
    • ns_server
    • Security Level: Public
    • None

    Description

      build: 1409

      issues was found when test cluster with ~10,000,000 items with rebalance in/out( mcsoda was used for data ops)

      approximate steps:
      10.3.121.112
      10.3.121.113
      10.3.121.114
      10.3.121.115
      10.3.121.116
      1.one node 10.3.121.112
      2. upload data pytests/performance/mcsoda.py membase://10.3.121.112:8091 max-items=10000000 json=1 doc-gen=0 doc-cache=0 ratio-sets=0.9 vbuckets=1024
      3.rebalance in 10.3.121.113, 10.3.121.114,10.3.121.115 during data loading incrementally
      4.stop data loading when number items is 11105199
      5.rebalance out 10.3.121.115
      6. 10% delete , 100% update data pytests/performance/mcsoda.py membase://10.3.121.112:8091 max-creates=10000000 json=1 doc-gen=0 doc-cache=0 ratio-sets=0 ratio-deletes=0.1 vbuckets=1024
      7. rebalance in 10.3.121.115, 10.3.121.116 ( #6 is performed)
      8.during rebalancing added production view

      at one point the orchestrator ceased to display statistics. all other nodes display stats partially
      see screenshot

      exception in the logs:

      [couchdb:info] [2012-07-05 19:11:39] [ns_1@10.3.121.112:<0.5714.19>:couch_log:info:39] Updater checkpointing set view `default` update for main group `_design/dev_AA`
      [stats:error] [2012-07-05 19:11:42] [ns_1@10.3.121.112:<0.7243.0>:stats_collector:handle_info:96] Exception in stats collector: {exit,
      {timeout,
      {gen_server,call,
      [

      {'couch_stats_reader-default', 'ns_1@10.3.121.112'}

      ,
      fetch_stats]}},
      [

      {gen_server,call,2}

      ,

      {couch_stats_reader,fetch_stats,1}

      ,

      {stats_collector,grab_all_stats,1}

      ,

      {stats_collector,handle_info,2}

      ,

      {gen_server,handle_msg,5}

      ,

      {proc_lib,init_p_do_apply,3}

      ]}

      Attachments

        1. 10.3.121.112-8091-diag.txt.gz
          7.89 MB
        2. 10.3.121.113-8091-diag.txt.gz
          6.25 MB
        3. 10.3.121.114-8091-diag.txt.gz
          4.06 MB
        4. 10.3.121.115-8091-diag.txt.gz
          3.21 MB
        5. 10.3.121.116-8091-diag.txt.gz
          2.27 MB
        6. stats.png
          stats.png
          193 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            alkondratenko Aleksey Kondratenko (Inactive)
            andreibaranouski Andrei Baranouski
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty