Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44264

metakv performance measurements

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • None
    • ns_server
    • None
    • 1

    Description

      Dave Finlay This is the MB I mentioned to you in Slack 2021-02-10 Jeelan Poola asked me to open to collect metakv performance measurements I did for MB-43285 in an MB for ns_server to look at.

      Summary of findings (these were all from GSI delete tokens, which are ~200 bytes each):

      1. 1us = per-tombstone incremental cost of stepping over delete token tombstones (in metakv.ListAllChildren("/indexing/ddl/commandToken/delete/") API)
      2. 1ms = per delete token incremental cost of listing live delete tokens (from same API call)
      3. 20ms = per delete token cost of retrieving one delete token (metakv.Get() API)

      Thus tombstones are 3 orders of magnitude faster than listing a token and 4 orders of magnitude faster than retrieving a token.

      Discussion with you today uncovered that metkav.ListAllChildren() actually retrieves not just the list of keys but also all the values of those keys (i.e. all the tokens, in GSI parlance). GSI code evidently did not realize this, as it has been first listing all the tokens via metakv.ListAllChildren() and then iterating over the paths returned by that call to retrieve each token one by one via metakv.Get(). In fact the tokans are already present as the Value []byte slices in the entries returned by metakv.ListAllChildren(). (I have verified this empirically.)

      For #1-2 I will attach graphs and the raw data point files. #3 is derived from the experiment for #2 (details below).

      Attachments

        1. list1000
          19 kB
        2. list1000.png
          list1000.png
          53 kB
        3. mb43285.sh
          0.3 kB
        4. tombstones72500
          9 kB
        5. tombstones72500.excl_8_outliers
          9 kB
        6. tombstones72500.excl_8_outliers.png
          tombstones72500.excl_8_outliers.png
          33 kB
        7. tombstones72500.excl_8_outliers.short.png
          tombstones72500.excl_8_outliers.short.png
          29 kB
        8. tombstones72500.png
          tombstones72500.png
          31 kB

        Activity

          People

            dfinlay Dave Finlay
            kevin.cherkauer Kevin Cherkauer (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              PagerDuty