Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7634

items lost after doing an offline upgrade of a cluster which has xdcr replication stream to another 2.0 cluster

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0, 2.0.1
    • Fix Version/s: 2.0, 2.0.1
    • Component/s: installer
    • Security Level: Public
    • Labels:
      None
    • Environment:
      Ubuntu / Centos

      Description

      Scenario:

      • 5:5 ec2 node ubuntu 12.04 clusters
      • Hostnames used, ip file updated with hostname
      • biXDCR_bucket: C1 <---> C2
      • uniXDCR: C1 ----> C2
      • Load about a million items on both buckets at C1, and another million items on just the biXDCR_bucket on C2
      • Initiate offline upgrade on C1 (from 2.0 to 2.0.1)
      • Stop all nodes on C1.
      • Manually upgrade the cluster (dpkg -i couchbase-server... )
      • Upgrade completes, buckets recreated, however items not loaded
      • Also XDCR replications not retrieved.

      The live cluster:
      C1: http://ec2-177-71-230-72.sa-east-1.compute.amazonaws.com:8091/
      C2: http://ec2-175-41-186-167.ap-southeast-1.compute.amazonaws.com:8091/

      Access:
      ssh -i SanPauloQAKey.pem ubuntu@ec2-177-71-230-72.sa-east-1.compute.amazonaws.com
      ssh -i SingaporeQAkey.pem ubuntu@ec2-175-41-186-167.ap-southeast-1.compute.amazonaws.com

      The items seen on biXDCR_bucket in C1, are those that were replicated from C2 to C1 after the upgrade of C1.

      1. SanPauloQAKey.pem
        2 kB
        Abhinav Dangeti
      2. SingaporeQAkey.pem
        2 kB
        Abhinav Dangeti
      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-7634=message:MB-7634

        Activity

        Hide
        abhinav Abhinav Dangeti added a comment -

        Shashank tried that scenario (without hostnames), replications were retrieved upon the offline upgrade.

        Show
        abhinav Abhinav Dangeti added a comment - Shashank tried that scenario (without hostnames), replications were retrieved upon the offline upgrade.
        Hide
        Aliaksey Artamonau Aliaksey Artamonau added a comment -

        It seems that we don't correctly restore /opt/couchbase/etc/couchdb/local.ini file after upgrade. On ec2-177-71-230-72.sa-east-1.compute.amazonaws.com I can see that there's a local.ini.debsave file with correct database_dir in it. Whereas local.ini doesn't have it set at all. Thus the default value from default.ini is used.

        Show
        Aliaksey Artamonau Aliaksey Artamonau added a comment - It seems that we don't correctly restore /opt/couchbase/etc/couchdb/local.ini file after upgrade. On ec2-177-71-230-72.sa-east-1.compute.amazonaws.com I can see that there's a local.ini.debsave file with correct database_dir in it. Whereas local.ini doesn't have it set at all. Thus the default value from default.ini is used.
        Show
        bcui Bin Cui (Inactive) added a comment - http://review.membase.org/#/c/24390/
        Hide
        abhinav Abhinav Dangeti added a comment -

        Successfully upgraded from 2.0.0-1976-rel to 2.0.1-148-rel.
        Items retrieved, so were the replications after the upgrade.

        Show
        abhinav Abhinav Dangeti added a comment - Successfully upgraded from 2.0.0-1976-rel to 2.0.1-148-rel. Items retrieved, so were the replications after the upgrade.
        Hide
        shashank Shashank Gupta added a comment -

        Verified the upgrade from 2.0.0-1976-rel to 2.0.1-150-rel on windows.

        Show
        shashank Shashank Gupta added a comment - Verified the upgrade from 2.0.0-1976-rel to 2.0.1-150-rel on windows.

          People

          • Assignee:
            abhinav Abhinav Dangeti
            Reporter:
            abhinav Abhinav Dangeti
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes