Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4847

Show replica index progress for initial run

    Details

      Description

      Loaded 5 million docs. Then created 20 design docs (each with a single view) and attempted to query all 20. This triggered 4 replica index threads that blocked the main indexer's responsible for returning data and updating UI (since 4 is the default number of parallel_indexers). Depending on environment one option is to increase max_paralell_indexers as outlined here (MB-4831) and/or consolidate views to fewer design docs.

      Currently, the only way to know that replica indexers are running and blocking results is via the following couch call:

      http://10.2.2.31:8092/_active_tasks

      [

      {"pid":"<0.18927.4>","changes_done":1293938,"design_documents":["_design/dev_ddoc3"],"indexer_type":"replica","progress":52,"set":"default","started_on":1330483699,"total_changes":2473631,"type":"indexer","updated_on":1330489820}

      ,

      {"pid":"<0.16970.7>","changes_done":1507224,"design_documents":["_design/dev_ddoc5"],"indexer_type":"replica","progress":49,"set":"default","started_on":1330486613,"total_changes":3046417,"type":"indexer","updated_on":1330489866}

      ,

      {"pid":"<0.31283.8>","changes_done":142396,"design_documents":["_design/dev_ddoc6"],"indexer_type":"replica","progress":4,"set":"default","started_on":1330488174,"total_changes":3263440,"type":"indexer","updated_on":1330489864}

      ,

      {"pid":"<0.15265.9>","changes_done":185568,"design_documents":["_design/dev_ddoc7"],"indexer_type":"replica","progress":5,"set":"default","started_on":1330489741,"total_changes":3263440,"type":"indexer","updated_on":1330489867}

      ,

      {"pid":"<0.16567.9>","changes_done":6087,"database":"default/73","progress":24,"started_on":1330489863,"total_changes":24557,"type":"database_compaction","updated_on":1330489868}

      ]

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        tommie Tommie McAfee added a comment - - edited

        When speaking with Fillipe about this, he suggested to display progress in the UI during initial building of replica index as this is the only time in which no results are returned to user since active indexers are blocked.

        Show
        tommie Tommie McAfee added a comment - - edited When speaking with Fillipe about this, he suggested to display progress in the UI during initial building of replica index as this is the only time in which no results are returned to user since active indexers are blocked.
        Hide
        FilipeManana Filipe Manana (Inactive) added a comment -

        Tommie, I think this is not necessary once http://www.couchbase.com/issues/browse/MB-4848 is in place.

        I'm thinking about allowing 4 main indexers in parallel (max_parallel_indexers) and 2 replica indexers in parallel (max_parallel_replica_indexers)
        These are configurable of course. By having max_parallel_indexers > max_parallel_replica_indexers , the user always sees progress.

        Show
        FilipeManana Filipe Manana (Inactive) added a comment - Tommie, I think this is not necessary once http://www.couchbase.com/issues/browse/MB-4848 is in place. I'm thinking about allowing 4 main indexers in parallel (max_parallel_indexers) and 2 replica indexers in parallel (max_parallel_replica_indexers) These are configurable of course. By having max_parallel_indexers > max_parallel_replica_indexers , the user always sees progress.
        Hide
        FilipeManana Filipe Manana (Inactive) added a comment -

        Closing this.
        The change mentioned in the last comment, was applied to DP4.

        Show
        FilipeManana Filipe Manana (Inactive) added a comment - Closing this. The change mentioned in the last comment, was applied to DP4.

          People

          • Assignee:
            FilipeManana Filipe Manana (Inactive)
            Reporter:
            tommie Tommie McAfee
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes