Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44408

[BP MB-44269 6.6.2] - Performance: metakv API misunderstanding in GSI code: ListAllChildren() returns keys AND values

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      GSI code treats the ns_server metakv.ListAllChildren() API as if it returns only a list of all the keys in a directory, so if GSI also wants the values (tokens), it iterates over the entries returned by ListAllChildren() and calls metakv.Get(entry.Path) to retrieve each one, one by one. However, metakv.ListAllChildren() actually returns all the keys AND all their values, so the additional metakv.Get() calls, which cost about 20ms each, are not needed. The values are in the Value []byte slice fields of the returned entries and just need to be unmarshalled into the correct token type. I have verified this empirically.

      This is a follow-on to GSI MB-43285 (metakv performance findings of that are also summarized in ns_server MB-44264).

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            jeelan.poola Jeelan Poola added a comment -

            Mihir KamdarWayne Siu This is an optimisation identified in 7.0 which will significantly benefit 6.6.x as well, especially when there are large #of indexes (eliminates unnecessary calls to metakv). No functional impact. Request inclusion in 6.6.2. Thank you!

            jeelan.poola Jeelan Poola added a comment - Mihir Kamdar Wayne Siu This is an optimisation identified in 7.0 which will significantly benefit 6.6.x as well, especially when there are large #of indexes (eliminates unnecessary calls to metakv). No functional impact. Request inclusion in 6.6.2. Thank you!

            Wayne Siu Can you take a call on this? From functional testing perspective, I am fine - we have some tests that can exercise this.

            mihir.kamdar Mihir Kamdar (Inactive) added a comment - Wayne Siu Can you take a call on this? From functional testing perspective, I am fine - we have some tests that can exercise this.

            Build couchbase-server-6.6.2-9541 contains indexing commit 890fb5e with commit message:
            MB-44408 Use tokens returned by metakv.ListAllChildren

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9541 contains indexing commit 890fb5e with commit message: MB-44408 Use tokens returned by metakv.ListAllChildren

            Kevin Cherkauer - Requesting you to dev-verify this BP ticket of (MB-44269) .

            prajwal.kirankumar Prajwal‌ Kiran Kumar‌ (Inactive) added a comment - Kevin Cherkauer  - Requesting you to dev-verify this BP ticket of ( MB-44269 ) .

            Dev verified at Prajwal‌ Kiran Kumar‌'s request.

            kevin.cherkauer Kevin Cherkauer (Inactive) added a comment - - edited Dev verified at Prajwal‌ Kiran Kumar‌ 's request.

            People

              wayne Wayne Siu
              jeelan.poola Jeelan Poola
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty