Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-15254

Indexing with compaction, indexing 20M items takes at least 7-8 hours

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 4.0.0
    • 4.0.0
    • forestdb, secondary-index
    • Security Level: Public
    • Build 3146
    • Untriaged
    • Yes
    • ForestDB: Oct 26 - Nov 13

    Description

      Appears to be a regression. I first saw the issue when testing build 3138. Note that the logs are filled with the following error message.

      2015-06-09T21:24:29.811Z-07:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 124710 Milliseconds Vbucket 373

      The indexer node is 172.23.100.45. I didn't let the test run fully, but it was attempting to index items for about 1.5 to 2 hours. The previous test I ran I cancelled after the indexing hadn't completed after 7 hours. Note that indexing does make progress and doesn't appear to stop completely. Also, changing the log level to error doesn't seem to help either.

      Indexer settings:

      {
        "indexer.settings.bufferPoolBlockSize": 16384,
        "indexer.settings.compaction.check_period": 30,
        "indexer.settings.compaction.interval": "00:00,00:00",
        "indexer.settings.compaction.min_frag": 30,
        "indexer.settings.compaction.min_size": 1048576,
        "indexer.settings.cpuProfFname": "",
        "indexer.settings.cpuProfile": false,
        "indexer.settings.inmemory_snapshot.interval": 200,
        "indexer.settings.largeSnapshotThreshold": 200,
        "indexer.settings.log_level": "error",
        "indexer.settings.maxVbQueueLength": 10000,
        "indexer.settings.max_cpu_percent": 400,
        "indexer.settings.memProfFname": "",
        "indexer.settings.memProfile": false,
        "indexer.settings.memory_quota": 104857600000,
        "indexer.settings.persisted_snapshot.interval": 30000,
        "indexer.settings.recovery.max_rollbacks": 5,
        "indexer.settings.scan_timeout": 120000,
        "indexer.settings.send_buffer_size": 1024,
        "indexer.settings.sliceBufSize": 50000,
        "indexer.settings.statsLogDumpInterval": 60,
        "projector.settings.log_level": "error",
        "queryport.client.settings.poolOverflow": 30,
        "queryport.client.settings.poolSize": 1000
      }
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            deepkaran.salooja Deepkaran Salooja
            mikew Mike Wiederhold [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty