Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-8319

[Doc'd] View compaction is triggered during rebalance when auto-compaction is disabled

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 2.0.1, 2.1.0
    • Fix Version/s: 3.0
    • Component/s: ns_server
    • Security Level: Public
    • Labels:
      None
    • Triage:
      Untriaged
    • Operating System:
      Centos 64-bit

      Description

      Steps:
      1. Set fragmentation threshold to 100% (both data and index)
      2. Start workload, view fragmentation is high, >90%.
      3. Rebalance cluster

      As result view compaction kicks in. Bucket compaction doesn't kick in.

      Not sure if it's a bug but behavior is very confusing. At least I didn't find documentation that explains it.

      Diags for 2.0.2, just in case:
      http://172.23.96.10:8080/view/views/job/apollo-views/116/artifact/

      Basically I just want to run a test with no view compaction.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        I'm not sure at all it's a bug and thus worth fixing

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - I'm not sure at all it's a bug and thus worth fixing
        Hide
        kzeller kzeller added a comment -

        Added to RN and cross ref to REST chapter:

        Indexing and Querying

        If you rebalance a cluster, index compaction starts but bucket compaction will not start. This is because vBucket cleanup from an index is not accounted for in index fragmentation. After this cleanup process completes it will cause more index fragmentation and therefore we run compaction again. Index compaction will therefore always run after a certain number of changes to vBuckets on nodes. You can change this setting using the REST-API, see Section 8.7.7, “Adjusting Rebalance during Compaction”.

        Issues: MB-8319

        REST Chapter content:

        http://www.couchbase.com/docs/couchbase-manual-2.1.0/couchbase-admin-restapi-rebalance-before-compaction.html

        Removing "doc" component until future release.

        Show
        kzeller kzeller added a comment - Added to RN and cross ref to REST chapter: Indexing and Querying If you rebalance a cluster, index compaction starts but bucket compaction will not start. This is because vBucket cleanup from an index is not accounted for in index fragmentation. After this cleanup process completes it will cause more index fragmentation and therefore we run compaction again. Index compaction will therefore always run after a certain number of changes to vBuckets on nodes. You can change this setting using the REST-API, see Section 8.7.7, “Adjusting Rebalance during Compaction”. Issues: MB-8319 REST Chapter content: http://www.couchbase.com/docs/couchbase-manual-2.1.0/couchbase-admin-restapi-rebalance-before-compaction.html Removing "doc" component until future release.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Eventually things (especially indexes during rebalance) will be a totally different approach. So imho not worth keeping this open.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Eventually things (especially indexes during rebalance) will be a totally different approach. So imho not worth keeping this open.
        Hide
        pavelpaulau Pavel Paulau added a comment -

        could we create a separate task for documentation and keep this open? imho we should change this behavior eventually.

        Show
        pavelpaulau Pavel Paulau added a comment - could we create a separate task for documentation and keep this open? imho we should change this behavior eventually.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        I have already replied above. Let me know if that's not enough.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - I have already replied above. Let me know if that's not enough.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        The reason is that it's known that cleanup vbuckets (those that we're removing from index) are not properly accounted for in index fragmentation. After background cleanup is complete that stuff will be deleted (causing more fragmentation) after which compaction will need another pass. So our logic is to do view compaction after certain number of incoming or outgoing moves from certain nodes regardless of anything. This can be disabled via internal settings.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - The reason is that it's known that cleanup vbuckets (those that we're removing from index) are not properly accounted for in index fragmentation. After background cleanup is complete that stuff will be deleted (causing more fragmentation) after which compaction will need another pass. So our logic is to do view compaction after certain number of incoming or outgoing moves from certain nodes regardless of anything. This can be disabled via internal settings.

          People

          • Assignee:
            anil Anil Kumar
            Reporter:
            pavelpaulau Pavel Paulau
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes