Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-13582

Excessive time to build Primary Index for large datasets (100M)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • 4.0.0
    • 4.0.0
    • secondary-index
    • Security Level: Public
    • None
    • 6 node CentOS. 4 Data, 1 Query, 1Indexer. each node 48vCPU, 256GRam, Raid10 2xSSD.
      100M 1K document load & indexing for N1QL query access

    Description

        • test rerun with "CREATE PRIMARY INDEX ON `bucket-1` USING GSI". primary index shows 'not online' after 6+ hours. logs are replace and from "USING GSI" condition.

      data load proceeded normally. used 'cbq' to issue "CREATE PRIMARY INDEX ON `bucket-1`
      can see from console "design/ddl#parimary/_view/#primary" gets created. but at 2.5 hrs, it looks like it is only ~1/3 done. looking on the query and index nodes the systems are lightly used, <2% CPU, <10%memory, and the the 4 data nodes show at the console only avg 3K Ops/sec. there is no other activity that CREATE PRIMARY INDEX.

      hoping there are configuration settings that might improve this situation. as is, have backed off 100M doct to 20M doct which seems to be pretty excessive, but doable. will upload cbcollect_info for all nodes

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-13636 Cluster intercommunication indicating projector connection breaks/restarts

          • Test Blocker - QE-related tests can't proceed
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-13603 Excessive time to build Primary Index using gsi for large datasets

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              thomas Thomas Anderson (Inactive)
              thomas Thomas Anderson (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty