Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6629

Slow incremental indexing and view compaction on idle cluster

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 2.0
    • 2.0-beta
    • view-engine
    • Security Level: Public
    • 1695

    Description

      Workload:-
      Phase1
      Load 15M items, 1K value sized , wait for indexing and view compaction to complete,
      It took about ~73 minutes to create the first index.
      Phase2
      Mutate 10M items, and change value size to 1.8K.
      Wait for indexing and compaction to complete.
      There is no front end load, the cluster is just doing view compaction.
      The view compaction post initial building process is still going on. ~ 48hrs plus

      1 ddoc with 4 views in each.
      15M items, replica=1, replica_index=enabled.

      1 ddocs, 4 views in each.
      10.6.2.7:8091

      Indexes and database files live separately, with dedicated disks.

      Some disk benchmarks on index partition:-
      https://friendpaste.com/1JxnXjI6YcsQtFLSAWqYa6
      https://friendpaste.com/1JxnXjI6YcsQtFLSAWqXsO

      node 7
      8.5G /data2/@indexes/default/main_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.2
      7.7G /data2/@indexes/default/replica_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.1
      6.1G /data2/@indexes/default/replica_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.1.compact
      257M /data2/@indexes/default/replica_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.log
      20K /data2/@indexes/default/tmp_02dcc6636d5e6f2bb68fd0fc592ba9a0_main
      16K /data2/@indexes/default/tmp_02dcc6636d5e6f2bb68fd0fc592ba9a0_replica

      node 11
      11G /data2/@indexes/default/main_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.1
      8.9G /data2/@indexes/default/main_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.1.compact
      565M /data2/@indexes/default/main_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.log
      8.8G /data2/@indexes/default/replica_02dcc6636d5e6f2bb68fd0fc592ba9a0.view.1
      20K /data2/@indexes/default/tmp_02dcc6636d5e6f2bb68fd0fc592ba9a0_main
      20K /data2/@indexes/default/tmp_02dcc6636d5e6f2bb68fd0fc592ba9a0_replica

      Attachments

        1. 10.6.2.11-9122012-diag.zip
          14.93 MB
        2. 10.6.2.11-9132012-diag.zip
          3.97 MB
        3. 10.6.2.7-9122012-diag.zip
          15.17 MB
        4. 10.6.2.7-9132012-diag.zip
          4.57 MB
        5. pidstat_node7.log
          61 kB
        6. pidstats_node11.log
          65 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            karan Karan Kumar (Inactive)
            karan Karan Kumar (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty