Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-36261

[System Test] DCP_DELETION:status:failure:duplicate item when vbstate is non-activ

    XMLWordPrintable

Details

    Description

      Build : 6.5.0-4380
      Test : -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml
      Scale : 3

      On nodes 172.23.108.103 and 172.23.97.121, the following type of errors are seen in the memcached logs.

      2019-10-01T15:38:06.230443-07:00 INFO (default) DCP (Producer) eq_dcpq:cbas:default:a15fc5822127736188f5b74e88f38d51:12 - Notifying paused connection now that DcpProducer::BufferLog is no longer full; ackedBytes:184359238, bytesSent:103654, maxBytes:131072
      		 
      2019-10-01T15:38:06.251965-07:00 ERROR 99: exception occurred in runloop during packet execution. Cookie info: [{"aiostat":"success","connection":"[ 127.0.0.1:56389 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]","engine_storage":"0x00007f8ae9fcc910","ewouldblock":false,"packet":{"bodylen":40,"cas":1569969486233468928,"datatype":"raw","extlen":21,"key":"<ud>.B84DB191-91_618495</ud>","keylen":19,"magic":"ClientRequest","opaque":17,"opcode":"DCP_DELETION","vbucket":49},"refcount":1}] - closing connection ([ 127.0.0.1:56389 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]): CheckpointManager::queueDirty(vb:49) - got Ckpt::queueDirty() status:failure:duplicate item when vbstate is non-active:2
      		 
      2019-10-01T15:38:06.252047-07:00 INFO 99: (No Engine) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - Removing connection [ 127.0.0.1:56389 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]
      		 
      2019-10-01T15:38:06.252077-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:44) Setting stream to dead state, last_seqno is 91014, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252094-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:45) Setting stream to dead state, last_seqno is 70159, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252109-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:46) Setting stream to dead state, last_seqno is 50228, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252123-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:51) Setting stream to dead state, last_seqno is 50058, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252142-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:601) Setting stream to dead state, last_seqno is 50054, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252157-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:48) Setting stream to dead state, last_seqno is 70130, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252171-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:49) Setting stream to dead state, last_seqno is 80662, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252185-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:50) Setting stream to dead state, last_seqno is 70559, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:06.252201-07:00 WARNING 99: (default) DCP (Consumer) eq_dcpq:replication:ns_1@172.23.97.239->ns_1@172.23.108.103:default - (vb:47) Setting stream to dead state, last_seqno is 50037, unAckedBytes is 0, status is The stream closed early because the conn was disconnected
      		 
      2019-10-01T15:38:08.139226-07:00

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MB-36255 [System Test]: Rebalance failed processing DCP_DELETION: PassiveDurabilityMonitor::completeSyncWrite No tracked, but received commit for key

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          For Gerrit Dashboard: MB-36261
          # Subject Branch Project Status CR V
          115757,4 MB-36261: Do not erase keys from indexes of open ckpts when expelling master kv_engine Status: MERGED +2 +1

          Activity

            People

              girish.benakappa Girish Benakappa
              girish.benakappa Girish Benakappa
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty