Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46050

(Index reset on rollback) Partitioned Indexes online stay at state "deferred" for a long time after flushing bucket (2000 indexes)

    XMLWordPrintable

Details

    Description

      Steps to reproduce
      1. Create 2 node init cluster kv, n1ql
      2. Create a bucket of total 1k collections. Each collection has 10 documents
      3. Rebalance-in 18 index nodes
      4. Rebalance-out 14 index nodes. So 4 indexes nodes are now in the cluster.
      5. Create 2K partitioned indexes (no replicas, and partitioned by hash) such that there are 2 indexes created per collection. Each index is indexed with 10 documents now with defer_build=true. Build them after they all get deferred. 
      Total online indexes count is 2K at this point.
      6. Rebalance-in 14 index nodes
      Rebalance completes in close to 10 hours.
      7. Flush the entire bucket.
      Now the indexes go to state "deferred" (from online) for a long time. Indexes UI page shows that "created" as against to "ready".

      SELECT count(*) FROM system:indexes where state = 'deferred'

      gives 2000
      Eventually after some time (approx ~1 hour I think), they started getting rebuilt.

      Logs attached. 

      Attachments

        1. after_flush.png
          after_flush.png
          393 kB
        2. before_flush.png
          before_flush.png
          428 kB
        3. rebalance_and_flush.png
          rebalance_and_flush.png
          545 kB
        4. servers.png
          servers.png
          489 kB

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. MB-47544 Reset Index on storage corruption

          • Critical - Crashes, loss of data, severe memory leak.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-46124 review rstate for partitioned index during index reset

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MB-46125 Enable index reset on rollback after MB-46050 is complete

          • Critical - Crashes, loss of data, severe memory leak.
          • Open
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              kevin.cherkauer Kevin Cherkauer (Inactive)
              sumedh.basarkod Sumedh Basarkod (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty