Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4937

ep-engine provides disk fragmentation statistics

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.2, 1.8.0
    • Fix Version/s: 1.8.1
    • Component/s: None
    • Security Level: Public

      Description

      Currently, there is no easy way to understand the disk fragmentation of SQLite. Expose this stat that should be calculated in ep_engine and exposed via REST API and the UI to better understand SQLite fragmentation over time so that users can understand when maintenance of the system using swap rebalance, vacuuming or other methods is required.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        chiyoung Chiyoung Seo added a comment -

        These timings show the distribution of disk head seek distance that have been performed on every read / write operation on Sqlite. We expect that these seek distance would be increased over the time as the sqlite is getting fragmented more and more.

        Show
        chiyoung Chiyoung Seo added a comment - These timings show the distribution of disk head seek distance that have been performed on every read / write operation on Sqlite. We expect that these seek distance would be increased over the time as the sqlite is getting fragmented more and more.
        Hide
        dipti Dipti Borkar added a comment -

        Chiyoung, can you provide some guidelines about what these mean, and how to interpret them so that we can add to the documentation.

        Show
        dipti Dipti Borkar added a comment - Chiyoung, can you provide some guidelines about what these mean, and how to interpret them so that we can add to the documentation.
        Hide
        chiyoung Chiyoung Seo added a comment -

        The stat subkey is "kvtimings". For example, the following shows how we can grab read/write seek histograms from the underlying kvstore:

        Chiyoung-MacBook:ep-engine chiyoung$ ./management/cbstats 127.0.0.1:11210 kvtimings
        readSeek (23 total)
        0 - 2 : ( 21.74%) 5 ###########################
        8 - 16 : ( 26.09%) 1 #####
        64 - 128 : ( 47.83%) 5 ###########################
        256 - 512 : ( 73.91%) 6 #################################
        512 - 1KB : ( 78.26%) 1 #####
        1KB - 2KB : ( 95.65%) 4 ######################
        2KB - 4KB : (100.00%) 1 #####
        writeSeek (77 total)
        4 - 8 : ( 28.57%) 22 ####################################
        8 - 16 : ( 36.36%) 6 #########
        16 - 32 : ( 57.14%) 16 ##########################
        256 - 512 : ( 61.04%) 3 ####
        512 - 1KB : ( 84.42%) 18 #############################
        1KB - 2KB : ( 97.40%) 10 ################
        2KB - 4KB : (100.00%) 2 ###

        Show
        chiyoung Chiyoung Seo added a comment - The stat subkey is "kvtimings". For example, the following shows how we can grab read/write seek histograms from the underlying kvstore: Chiyoung-MacBook:ep-engine chiyoung$ ./management/cbstats 127.0.0.1:11210 kvtimings readSeek (23 total) 0 - 2 : ( 21.74%) 5 ########################### 8 - 16 : ( 26.09%) 1 ##### 64 - 128 : ( 47.83%) 5 ########################### 256 - 512 : ( 73.91%) 6 ################################# 512 - 1KB : ( 78.26%) 1 ##### 1KB - 2KB : ( 95.65%) 4 ###################### 2KB - 4KB : (100.00%) 1 ##### writeSeek (77 total) 4 - 8 : ( 28.57%) 22 #################################### 8 - 16 : ( 36.36%) 6 ######### 16 - 32 : ( 57.14%) 16 ########################## 256 - 512 : ( 61.04%) 3 #### 512 - 1KB : ( 84.42%) 18 ############################# 1KB - 2KB : ( 97.40%) 10 ################ 2KB - 4KB : (100.00%) 2 ###
        Hide
        steve Steve Yen added a comment -

        to add usage notes

        Show
        steve Steve Yen added a comment - to add usage notes
        Hide
        steve Steve Yen added a comment -

        ns_server can grab histogram from ep-engine and calculate average.

        Show
        steve Steve Yen added a comment - ns_server can grab histogram from ep-engine and calculate average.

          People

          • Assignee:
            chiyoung Chiyoung Seo
            Reporter:
            dipti Dipti Borkar
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes