Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19900

Hack ep-engine stats to allow compaction to run with current ns_server code

    XMLWordPrintable

Details

    • Task
    • Resolution: Won't Do
    • Major
    • None
    • 5.0.0
    • couchbase-bucket

    Description

      Idea courtesy of Aliaksey Artamonau.

      We should create an ep-engine patch for use in toy builds that allows us to run tests with compaction on using current ns_server compaction logic. Idea is:

      1. The vbucket in the shard with the lowest vbucket ID should return the db_data_size and db_file_size in the stats call from the values for the whole shard
      2. Every other vbucket in the shard should return db_data_size and db_file_size values indicating that these vbuckets have zero fragmentation

      In this way, ns_server will schedule compaction on the shard when it calls compaction on the "representative" vbucket.

      What's nice about this is the fact that there's a small amount of work to lie to ns_server in ep-engine but then we get the ability to test performance and functionality with compaction on. Otherwise the rewriting of the ns_server side logic to incorporate shards is relatively significant – at least it seems much greater than the patch required on the ep-engine side to fake out the db_data_size and db_file_size stats.

      Of course, this will only work for compaction in append-only mode, since when circular block reuse is turned on the thinking is that we will not invoke compaction at all based on fragmentation rations, but rather the plan is probably to invoke it periodically. However, I would like to test in the near term in append-only mode and it would be good to get this patch created.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            sriram Sriram Ganesan (Inactive)
            dfinlay Dave Finlay
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty