Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47503

[7.0.2 BP MB-46932] - Recovered instances do not compact

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Yes

    Description

      During recovery, swap() of recovered instance seems to not copy storeCtx, causing cfGetter to return gcFilter with wrong gc snapshot number intervals. Any page compact of a recovered instance will cause MVCC versions to pile up.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Build couchbase-server-7.0.2-6524 contains plasma commit 9167dae with commit message:
            MB-47503: Do not continue on checkpoint recovery after recovery is done

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.2-6524 contains plasma commit 9167dae with commit message: MB-47503 : Do not continue on checkpoint recovery after recovery is done

            Build couchbase-server-7.0.2-6524 contains plasma commit 90965bd with commit message:
            MB-47503: Do not pre-allcoate plasma instance for recovery

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.2-6524 contains plasma commit 90965bd with commit message: MB-47503 : Do not pre-allcoate plasma instance for recovery
            wayne Wayne Siu added a comment -

            Akhil Mundroy added a comment - 19/Aug/21 9:47 PM
            Verification Steps:

            1. Create collection, add documents, create and build index.
            2. kill indexer process.
            3. Delete all documents in collection.
            4. Monitor storage stats:
              1. compacts should increase
              2. lss_data_size should decrease
              3. total_records should tend to 0

             

            Without the fix, stats in steps (4.2) and (4.3) may not come down to expected levels due to compacts not clearing out items.

            wayne Wayne Siu added a comment - Akhil Mundroy  added a comment - 19/Aug/21 9:47 PM Verification Steps: Create collection, add documents, create and build index. kill indexer process. Delete all documents in collection. Monitor storage stats: compacts  should increase lss_data_size  should decrease total_records  should tend to 0   Without the fix, stats in steps (4.2) and (4.3) may not come down to expected levels due to compacts not clearing out items.
            sanjit.chauhan Sanjit Chauhan added a comment - - edited

            Wayne Siu: What is the correct build version here?

            There is no 7.0.2-6524 build version

            instead there is 7.0.2-6523.

             

            I tested in 7.0.2-6523 and still I can see the issue. Though I deleted all the documents in collection still total_records > 0.  

            cc: Akhil Mundroy

            sanjit.chauhan Sanjit Chauhan added a comment - - edited Wayne Siu : What is the correct build version here? There is no 7.0.2-6524 build version instead there is 7.0.2-6523.   I tested in 7.0.2-6523 and still I can see the issue. Though I deleted all the documents in collection still total_records > 0.   cc:  Akhil Mundroy

            John Liang: Sure, Executing the below steps to validate the issue:

            1. Create collection, add documents, create and build index.
            2. kill indexer process.
            3. Delete all documents in collection.
            4. Wait for 20+ mins. Performing mutations for the period
            5. Execute CompactAll against every indexerId
            6. Monitor storage stats:
              1. compacts should increase
              2. lss_data_size should have downward trend as compared to step 1
              3. total_records should have downward trend as compared to step 1
            sanjit.chauhan Sanjit Chauhan added a comment - John Liang : Sure, Executing the below steps to validate the issue: Create collection, add documents, create and build index. kill indexer process. Delete all documents in collection. Wait for 20+ mins. Performing mutations for the period Execute CompactAll against every indexerId Monitor storage stats: compacts  should increase lss_data_size  should have downward trend as compared to step 1 total_records  should have downward trend as compared to step 1
            sanjit.chauhan Sanjit Chauhan added a comment - - edited

            Tested the issue total_records are updating after compaction. total_records updating to mutated count value.

            i.e. Created and deleted 10k documents. Later added 100 docs.

            Total_records is showing 100 only.

            ===================================================

            Compacts stats also going up after compaction

            ====================================================

            lss_data_size count going down after compaction

             

            Marking the issue resolved

            sanjit.chauhan Sanjit Chauhan added a comment - - edited Tested the issue total_records are updating after compaction. total_records updating to mutated count value. i.e. Created and deleted 10k documents. Later added 100 docs. Total_records is showing 100 only. =================================================== Compacts stats also going up after compaction ==================================================== lss_data_size count going down after compaction   Marking the issue resolved
            jliang John Liang added a comment -

            sounds good. Make sure you save the test log as well as cbcollect.

            jliang John Liang added a comment - sounds good. Make sure you save the test log as well as cbcollect.

            People

              sanjit.chauhan Sanjit Chauhan
              jliang John Liang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty