Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48514

Rebalance out taking more than 16 hrs

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • 7.1.0
    • 7.1.0
    • fts
    • Untriaged
    • 1
    • Unknown

    Description

      Build: 7.1.0-1300
      Test: -test tests/fts/cheshire-cat/test_fts_clusterops_cheshire_cat_coll_crud_freetier.yml -scope tests/fts/cheshire-cat/scope_fts_cheshire_cat_free_tier.yml

      • Cluster with 3 nodes having kv,n1ql, search, index on all the nodes
      • Create 1 bucket, 100 scopes and 50 collections in each scopes
      • Created 500 indexes: one index (1 partition) on each collection
      • Create 2500 GSI indexes ( 5 on each collection)
      • Load documents on some of the collections
      • Run fts query workload
      • Kill fts on 172.23.106.242 and wait for 25 mins
      • Rebalance in 172.23.107.89 with fts service
      • Rebalance failed due OOM kill of fts node on 172.23.106.242
      • Kill fts on 172.23.106.243 and wait for 15 mins
      • Mutate and add some more docs
      • rebalance out 172.23.106.242

      Rebalance taking more than 16hrs and continuing.

      Logs:

      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1631977995/collectinfo-2021-09-18T151316-ns_1%40172.23.106.242.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1631977995/collectinfo-2021-09-18T151316-ns_1%40172.23.106.243.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1631977995/collectinfo-2021-09-18T151316-ns_1%40172.23.106.253.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1631977995/collectinfo-2021-09-18T151316-ns_1%40172.23.107.89.zip

      Attachments

        Activity

          People

            Sreekanth Sivasankaran Sreekanth Sivasankaran (Inactive)
            girish.benakappa Girish Benakappa
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty