Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-57043

[System test upgrade] :- failover + recovery - online upgrade hangs in fts

    XMLWordPrintable

Details

    • Bug
    • Resolution: User Error
    • Blocker
    • 7.2.0
    • 7.2.0
    • fts
    • 7.1.4-3601 -> 7.2.0-5235
    • Untriaged
    • Centos 64-bit
    • 0
    • Unknown

    Description

      Steps to Repro
      1. Run a longevity test on 7.1.4 for 2 days.

      ./sequoia -client 172.23.104.27:2375 -provider file:centos_pine.yml -test tests/integration/neo/test_neo.yml -scope tests/integration/neo/scope_neo_magma.yml -scale 3 -repeat 0 -log_level 0 -version 7.1.4-3601 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=604800 -show_topology=true
      

      2. Upgrade to 7.2.0-5235 using failover + recovery strategy.

      I saw this hanging in FTS rebalance for 15+ hours.
      Please note that I have already applied the following work around of disabling file based transfer for rebalance.

      balakumaran.g@Balakumarans-MacBook-Pro-2 scripts % curl -XPUT -H "Content-type:application/json" http://Administrator:password@172.23.96.122:8094/api/managerOptions -d '{"disableFileTransferRebalance": "true"}' 
      {"status":"ok"}
      

      We did not hit the issue during upgrade from 7.1.4-3601 -> 7.2.0-5324 and we did not have fts fix in 7.2.0-5325 implying that it is some kind of race.

      cbcollect_info attached.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Balakumaran.Gopal Balakumaran Gopal
              Balakumaran.Gopal Balakumaran Gopal
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty