Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44688

[cbstats] checkpoint point stats missing for few vbuckets for sometime

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • Yes

    Description

      Build: 7.0.0-4554

      Scenario:

      • 4 node cluster, couchbase bucket (replicas=2, bucket_ttl=100)
      • Load few docs into default collection and immediately trigger "cbstats checkpoint" command

      Observation:

      Seeing few vbucket's stats are missing in the command output.

      For instance on 104.log we can see the vb_978's num_items_for_persistence is missing.

       vb_976:num_items_for_persistence: 0
       vb_977:num_items_for_persistence: 0
       vb_979:num_items_for_persistence: 0
       vb_980:num_items_for_persistence: 0
       vb_981:num_items_for_persistence: 0
       vb_982:num_items_for_persistence: 0

       TAF test:

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.33423.ini num_items=1000,durability=PERSIST_TO_MAJORITY,sdk_timeout=30,GROUP=P1;durability,get-cbcollect-info=False,upgrade_version=7.0.0-4554 -t epengine.expiry_maxttl.ExpiryMaxTTL.test_maxttl_lesser_doc_expiry,replicas=2,maxttl=100,durability=PERSIST_TO_MAJORITY,upgrade_version=7.0.0-4554,nodes_init=4,GROUP=P1;durability,num_items=1000,sdk_timeout=30'*no* further _formatting_ is done here 

      Attachments

        1. 172.23.123.104.log
          994 kB
        2. 172.23.96.183.log
          993 kB
        3. 172.23.97.106.log
          991 kB
        4. 172.23.97.114.log
          991 kB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Marking this as regression since the same scenario has works fine with 4502 and 4476 builds

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - Marking this as regression since the same scenario has works fine with 4502 and 4476 builds
            jwalker Jim Walker added a comment -

            Please update with cbcollects

            jwalker Jim Walker added a comment - Please update with cbcollects
            ashwin.govindarajulu Ashwin Govindarajulu added a comment - Hi Jim Walker , please find the cbcollect logs here, https://cb-engineering.s3.amazonaws.com/cbstats_issue/collectinfo-2021-03-13T081437-ns_1%40172.23.100.22.zip https://cb-engineering.s3.amazonaws.com/cbstats_issue/collectinfo-2021-03-13T081437-ns_1%40172.23.100.27.zip https://cb-engineering.s3.amazonaws.com/cbstats_issue/collectinfo-2021-03-13T081437-ns_1%40172.23.100.28.zip https://cb-engineering.s3.amazonaws.com/cbstats_issue/collectinfo-2021-03-13T081437-ns_1%40172.23.100.33.zip Jenkins log: http://qa.sc.couchbase.com/job/test_suite_executor-TAF/96924/console
            jwalker Jim Walker added a comment -

            Logs are full of

            ./cbcollect_info_ns_1@172.23.100.28_20210313-081437/memcached.log:2021-03-13T00:14:54.440006-08:00 WARNING (default) CheckpointManager::addStats: An error occurred while adding stats: std::bad_alloc
            

            This explains why we are missing stats in the output.

            jwalker Jim Walker added a comment - Logs are full of ./cbcollect_info_ns_1@172.23.100.28_20210313-081437/memcached.log:2021-03-13T00:14:54.440006-08:00 WARNING (default) CheckpointManager::addStats: An error occurred while adding stats: std::bad_alloc This explains why we are missing stats in the output.
            jwalker Jim Walker added a comment - - edited Duplicate of https://issues.couchbase.com/browse/MB-44685

            Closing duplicate bug.

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - Closing duplicate bug.

            People

              ashwin.govindarajulu Ashwin Govindarajulu
              ashwin.govindarajulu Ashwin Govindarajulu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty