Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-29651

Data loss seen with graceful failover on 4.6.4

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • None
    • 4.6.4
    • couchbase-bucket, ns_server
    • None

    Description

      Steps to Repro
      1) Created a 4 node cluster of 4.6.4
      2) started 3M data population using cbc-pillowfight
      3) When 2 in progress started graceful failover on 4.6.4 node(172.23.98.196)
      Data loss was noticed after step 3.
      4) Installed 5.1.1-5716(172.23.98.196) on the node and did full recovery and rebalanced.

      One doc was missing. Key - 00000000000000948466. It belongs to vbucket 910.

      [root@CentOS7 ~]#  cbc hash -U  couchbase://172.23.98.195/before_upgrade_bucket -u Administrator -P password 00000000000000948466
      00000000000000948466: [vBucket=910, Index=3] Server: 172.23.98.196:11210, CouchAPI: http://172.23.98.196:8092/before_upgrade_bucket
      Replica #0: Index=1, Host=172.23.123.251:11210
      [root@CentOS7 ~]#  cbc-cat -U  couchbase://172.23.98.195/before_upgrade_bucket -u Administrator -P password 00000000000000948466
      00000000000000948466 The key does not exist on the server (0xd)
      [root@CentOS7 ~]# 
      

      However tried repeating the test with other 3 nodes that were remaining in the cluster, I was not able to reproduce the issue and did not notice any data loss.

      Live server available for debugging - http://172.23.98.196:8091/ui/index.html#!/logs

      Logs attached.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty