Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-55551

[CDC] History is maintained despite of history settings disabled on collections

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Major
    • None
    • 7.2.0
    • couchbase-bucket
    • 7.2.0-5161

    Description

      Steps To Recreate:
      1. Create a 3 node cluster
      2. Create a magma buckets with (history_retention_collection_default=true,bucket_history_retention_seconds=0,bucket_history_retention_bytes=96636764160) (vbuckets = 16, replicas = 2)
      3. Create five collections(total collections including default are six)
      4. Disbale history on all collections

      curl localhost:8091/pools/default/buckets/default/scopes/_default/collections/FunctionCollection1 -u Administrator:password -X PATCH -d history=false -w "\n"


      -bash-4.2# /opt/couchbase/bin/cbstats localhost:11210 -u Administrator -p password -b default collections
      		 
       0x0:0x0:collections_mem_used: 0
      		 
       0x0:0x0:data_size:            0
      		 
       0x0:0x0:history:              false
      		 
       0x0:0x0:items:                0
      		 
       0x0:0x0:name:                 _default
      		 
       0x0:0x0:ops_delete:           0
      		 
       0x0:0x0:ops_get:              0
      		 
       0x0:0x0:ops_store:            0
      		 
       0x0:0x0:scope_name:           _default
      		 
       0x0:0x8:collections_mem_used: 31511297
      		 
       0x0:0x8:data_size:            28905242
      		 
       0x0:0x8:history:              false
      		 
       0x0:0x8:items:                31221
      		 
       0x0:0x8:name:                 FunctionCollection1
      		 
       0x0:0x8:ops_delete:           0
      		 
       0x0:0x8:ops_get:              0
      		 
       0x0:0x8:ops_store:            1430958
      		 
       0x0:0x8:scope_name:           _default
      		 
       0x0:0x9:collections_mem_used: 28349831
      		 
       0x0:0x9:data_size:            28905242
      		 
       0x0:0x9:history:              false
      		 
       0x0:0x9:items:                31221
      		 
       0x0:0x9:name:                 FunctionCollection2
      		 
       0x0:0x9:ops_delete:           0
      		 
       0x0:0x9:ops_get:              0
      		 
       0x0:0x9:ops_store:            1416391
      		 
       0x0:0x9:scope_name:           _default
      		 
       0x0:0xa:collections_mem_used: 28571886
      		 
       0x0:0xa:data_size:            28905242
      		 
       0x0:0xa:history:              false
      		 
       0x0:0xa:items:                31221
      		 
       0x0:0xa:name:                 FunctionCollection3
      		 
       0x0:0xa:ops_delete:           0
      		 
       0x0:0xa:ops_get:              0
      		 
       0x0:0xa:ops_store:            1417433
      		 
       0x0:0xa:scope_name:           _default
      		 
       0x0:0xb:collections_mem_used: 27675008
      		 
       0x0:0xb:data_size:            28905242
      		 
       0x0:0xb:history:              false
      		 
       0x0:0xb:items:                31221
      		 
       0x0:0xb:name:                 FunctionCollection4
      		 
       0x0:0xb:ops_delete:           0
      		 
       0x0:0xb:ops_get:              0
      		 
       0x0:0xb:ops_store:            1418471
      		 
       0x0:0xb:scope_name:           _default
      		 
       0x0:0xc:collections_mem_used: 27221154
      		 
       0x0:0xc:data_size:            28905242
      		 
       0x0:0xc:history:              false
      		 
       0x0:0xc:items:                31221
      		 
       0x0:0xc:name:                 FunctionCollection5
      		 
       0x0:0xc:ops_delete:           0
      		 
       0x0:0xc:ops_get:              0
      		 
       0x0:0xc:ops_store:            1419513
      		 
       0x0:0xc:scope_name:           _default
      		 
       manifest_uid:                 10

      5 Create 100000 items in each of the non default collection.(doc size : 1024 bytes)
      6. Update all the docs created in above step (40 times)
      7. Though history is disabled on all collections, observed it's being maintained

      -bash-4.2# /opt/couchbase/bin/cbstats localhost:11210 -u Administrator -p password -b default vbucket-details 1 | grep -e history -e high_seqno
      		 
       vb_1:high_seqno:                          1113239
      		 
       vb_1:history_start_seqno:                 1
      		 
       
      		 
      -bash-4.2# /opt/couchbase/bin/magma_dump /data/default/magma.0 --kvstore 0 --tree seq --treedata | grep random_keys00000036604 | grep '"cid":8'
      		 
          {"Seqno":6850,"Sn":7983,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181478354714624,"bySeqno":6850,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":37950,"Sn":48468,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181512598519808,"bySeqno":37950,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":69050,"Sn":109979,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181543712522240,"bySeqno":69050,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":100122,"Sn":173609,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181574804570112,"bySeqno":100122,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":131250,"Sn":236145,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181605979062272,"bySeqno":131250,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":162350,"Sn":298591,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181637090377728,"bySeqno":162350,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":193450,"Sn":361109,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181668229873664,"bySeqno":193450,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":224550,"Sn":423444,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181699295510528,"bySeqno":224550,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":255607,"Sn":486531,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181730399289344,"bySeqno":255607,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":286724,"Sn":547932,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181761497169920,"bySeqno":286724,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":317850,"Sn":611340,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181792583778304,"bySeqno":317850,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":348949,"Sn":673248,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181823669534720,"bySeqno":348949,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":380012,"Sn":735591,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181854718459904,"bySeqno":380012,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":411150,"Sn":798253,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181885813719040,"bySeqno":411150,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":442225,"Sn":861074,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181916962193408,"bySeqno":442225,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":473314,"Sn":923949,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181948032090112,"bySeqno":473314,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":504450,"Sn":986428,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676181979109785600,"bySeqno":504450,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":535550,"Sn":1048157,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676182010237091840,"bySeqno":535550,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":566628,"Sn":1110198,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676182041294012416,"bySeqno":566628,"deleted":false,"valueSize":883},"op":"update","valueLen":934}
      		 
          {"Seqno":597741,"Sn":1172799,"key":{"Ascii":" random_keys00000036604","Hex":"0872616e646f6d5f6b6579733030303030303336363034","cid":8,"namespace":"default"},"logicalSize":0,"meta":{"CAS":1676182072376360960,"bySeqno":597741,"deleted":false,"valueSize":883},"op":"update","valueLen":934}

      QE-TEST:

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/temp_vol_3.ini bucket_storage=magma,bucket_ram_quota=256,init_loading=False,bucket_eviction_policy=fullEviction,rerun=False -t storage.magma.magma_crash_recovery.MagmaCrashTests.test_magma_failures_with_CDC,nodes_init=3,skip_cleanup=True,num_items=100000,doc_size=1024,batch_size=1000,sdk_timeout=60,log_level=info,infra_log_level=info,key_size=22,num_collections=6,ops_rate=20000,key_type=SimpleKey,vbuckets=16,replicas=2,doc_ops=update,test_itr=50,bucket_history_retention_seconds=0,bucket_history_retention_bytes=96636764160,standard_buckets=1,magma_buckets=1,num_scopes=1,induce_failures=False,history_for_collections=False'

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ankush.sharma Ankush Sharma
            ankush.sharma Ankush Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty