Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-15625

Investigate scaling behavior for incremental index build when shared within the same node

    XMLWordPrintable

Details

    • Task
    • Resolution: Won't Do
    • Critical
    • 4.5.0
    • 4.0.0
    • performance
    • Security Level: Public
    • None

    Description

      On a 4KV + 1 indexer cluster, if I add another indexer (physical machine) and shard/partition the index over the two indexer nodes (each indexer node gets one index with a predicate that cuts the index's key space in half) vs. putting these two indexes on the same indexer box (sharding within one indexer node), we see the following results compared to baseline. Note, 53% here means it takes 53% of the original/baseline time.

      initial build Incremental build
      2 indexers 53.27% 61.51%
      1 indexer 2 shards 62.31% 81.20%

      link to full data: https://docs.google.com/document/d/1v620uzoWIGRFIZY4bdSzj5IYnpTRsstPtxw30GApJ5Q/edit?usp=sharing

      We should investigate why 81.20% isn't closer to 50%, ie., why doesn't it scale as linearly as the others?

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            dkao David Kao (Inactive)
            dkao David Kao (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty