Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7699

[system test] buckets replication appear in "Ongoing Replication" on new setup cluster

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0
    • Fix Version/s: 2.0.1
    • Component/s: XDCR
    • Security Level: Public
    • Labels:
    • Environment:
      centos 5.x 64bit

      Description

      Create new xdcr cluster.

      Environment:

      • Source:
        Each node has 4 core CPU, 4GB RAM
        Install couchbase server 2.0.0-1976 on 2 node cluster
        2 nodes cluster using host name (not IP), one node with default data path and other node with custom data path
        Create 2 bucket, one default (2GB) with one replica and one sasl (1.1GB) bucket with 2 replica
      • Destination:
        Each node has 4 core CPU, 4GB RAM
        Install couchbase server 2.0.0-1976 on 2 node cluster
        2 nodes cluster using IP, one node with default data path and other node with custom data path
        Create 2 bucket, one default (2GB) with one replica and one sasl (1.1GB) bucket with 2 replica

      At source cluster, create one doc with 3 views
      Buckets at source and destination are empty.4

      Looked at destination cluster in Ongoing Replication session, I saw replication of default and sasl bucket appear in it even though I did not create them. I delete them as it shows in log.

      Replication from bucket "sasl" to bucket "sasl" on cluster "unknown" removed. menelaus_web000 ns_1@10.3.3.7 13:07:47 - Wed Feb 6, 2013
      Replication from bucket "default" to bucket "default" on cluster "unknown" removed. menelaus_web000 ns_1@10.3.3.7 13:07:45 - Wed Feb 6, 2013

      Link to collect info files at source cluster:
      https://s3.amazonaws.com/packages.couchbase/collect_info/2_0_1/201302/2nodes-200GA-src-xdcr-bucket-shutdown-at-des-20130206-135530.tgz

      Link to collect info files at destination cluster:
      https://s3.amazonaws.com/packages.couchbase/collect_info/2_0_1/201302/2nodes-200GA-des-xdcr-bucket-shutdown-at-des-20130206-135530.tgz

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        junyi Junyi Xie (Inactive) added a comment -

        Tonny,

        At destination cluster where you see the phantom "ongoing replication", did you install the build from scratch or delete the bucket? These "ongoing replication" can be from previous test which was not cleaned up completely.

        Show
        junyi Junyi Xie (Inactive) added a comment - Tonny, At destination cluster where you see the phantom "ongoing replication", did you install the build from scratch or delete the bucket? These "ongoing replication" can be from previous test which was not cleaned up completely.
        Hide
        thuan Thuan Nguyen added a comment -

        Nodes from destination cluster were from clean install.

        Show
        thuan Thuan Nguyen added a comment - Nodes from destination cluster were from clean install.
        Hide
        junyi Junyi Xie (Inactive) added a comment -

        Tony,

        From destination log, the cluster started at around 2013-02-06T12:32:10

        But at around 12:54:42, somebody else apparently created an XDCR on your cluster, either on UI, or using some REST API

        [xdcr:debug,2013-02-06T12:54:42.782,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_manager:process_update:200]replication doc deleted (docId: <<"8b499e98e24bd5cc31933141efddc3dd/sasl/sasl">>), stop all replications
        [xdcr:debug,2013-02-06T12:54:42.783,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:stop_replication:75]all replications for DocId <<"8b499e98e24bd5cc31933141efddc3dd/sasl/sasl">> have been stopped
        [xdcr:debug,2013-02-06T12:54:42.784,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_manager:process_update:200]replication doc deleted (docId: <<"8b499e98e24bd5cc31933141efddc3dd/default/default">>), stop all replications
        [xdcr:debug,2013-02-06T12:54:42.791,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:stop_replication:75]all replications for DocId <<"8b499e98e24bd5cc31933141efddc3dd/default/default">> have been stopped
        [xdcr:debug,2013-02-06T12:54:42.791,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_manager:process_update:207]replication doc (docId: <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">>) modified, parse new doc and adjsut replications for change (source <<"sasl">>, target: <<"/remoteClusters/39fa7fb455b6b3d8ccfd2a3be7caff48/buckets/sasl">>)
        [xdcr:debug,2013-02-06T12:54:42.799,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_utils:make_options:164]Options for replication:worker processes: 4, worker batch size (# of mutations): 500, HTTP connections: 20, connection timeout (ms): 180000,num of retries per request: 2
        [xdcr:debug,2013-02-06T12:54:42.799,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:stop_replication:75]all replications for DocId <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">> have been stopped
        [xdcr:info,2013-02-06T12:54:42.799,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:start_replication:36]start bucket replicator using spec: {{<<"sasl">>,
        <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">>},
        {xdc_replication,start_link,
        [{rep,
        <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">>,
        <<"sasl">>,
        <<"/remoteClusters/39fa7fb455b6b3d8ccfd2a3be7caff48/buckets/sasl">>,
        [

        {connection_timeout,180000}

        ,

        {continuous,true}

        ,

        {http_connections,20}

        ,

        {retries,2}

        ,
        {socket_options,
        [

        {keepalive,true}

        ,

        {nodelay,false}

        ]},

        {worker_batch_size,500}

        ,

        {worker_processes,4}

        ]}]},
        permanent,100,worker,
        [xdc_replication]}.

        Show
        junyi Junyi Xie (Inactive) added a comment - Tony, From destination log, the cluster started at around 2013-02-06T12:32:10 But at around 12:54:42, somebody else apparently created an XDCR on your cluster, either on UI, or using some REST API [xdcr:debug,2013-02-06T12:54:42.782,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_manager:process_update:200] replication doc deleted (docId: <<"8b499e98e24bd5cc31933141efddc3dd/sasl/sasl">>), stop all replications [xdcr:debug,2013-02-06T12:54:42.783,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:stop_replication:75] all replications for DocId <<"8b499e98e24bd5cc31933141efddc3dd/sasl/sasl">> have been stopped [xdcr:debug,2013-02-06T12:54:42.784,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_manager:process_update:200] replication doc deleted (docId: <<"8b499e98e24bd5cc31933141efddc3dd/default/default">>), stop all replications [xdcr:debug,2013-02-06T12:54:42.791,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:stop_replication:75] all replications for DocId <<"8b499e98e24bd5cc31933141efddc3dd/default/default">> have been stopped [xdcr:debug,2013-02-06T12:54:42.791,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_manager:process_update:207] replication doc (docId: <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">>) modified, parse new doc and adjsut replications for change (source <<"sasl">>, target: <<"/remoteClusters/39fa7fb455b6b3d8ccfd2a3be7caff48/buckets/sasl">>) [xdcr:debug,2013-02-06T12:54:42.799,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_rep_utils:make_options:164] Options for replication: worker processes: 4, worker batch size (# of mutations): 500, HTTP connections: 20, connection timeout (ms): 180000,num of retries per request: 2 [xdcr:debug,2013-02-06T12:54:42.799,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:stop_replication:75] all replications for DocId <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">> have been stopped [xdcr:info,2013-02-06T12:54:42.799,ns_1@10.3.3.7:xdc_rep_manager<0.3766.0>:xdc_replication_sup:start_replication:36] start bucket replicator using spec: {{<<"sasl">>, <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">>}, {xdc_replication,start_link, [{rep, <<"39fa7fb455b6b3d8ccfd2a3be7caff48/sasl/sasl">>, <<"sasl">>, <<"/remoteClusters/39fa7fb455b6b3d8ccfd2a3be7caff48/buckets/sasl">>, [ {connection_timeout,180000} , {continuous,true} , {http_connections,20} , {retries,2} , {socket_options, [ {keepalive,true} , {nodelay,false} ]}, {worker_batch_size,500} , {worker_processes,4} ]}]}, permanent,100,worker, [xdc_replication] }.
        Hide
        junyi Junyi Xie (Inactive) added a comment -

        Can anybody please confirm it is really a bug?

        Show
        junyi Junyi Xie (Inactive) added a comment - Can anybody please confirm it is really a bug?
        Hide
        ketaki Ketaki Gangal added a comment -

        Cannot repro this on any other build.

        Show
        ketaki Ketaki Gangal added a comment - Cannot repro this on any other build.

          People

          • Assignee:
            ketaki Ketaki Gangal
            Reporter:
            thuan Thuan Nguyen
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes