Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7786

[RN 2.0.2] Frequent replication start error messages "Failed to grab remote bucket info, vbucket.." at start of replication.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.1.0
    • Component/s: ns_server
    • Security Level: Public
    • Environment:
      2.0.1.-160-rel
    • Flagged:
      Release Note

      Description

      Seeing 2 issues when trying to setup replication for the first time between two clusters -

      1.Frequently seeing error messages on creating the 1st time replication from cluster1 to cluster2.
      "Failed to grab remote bucket info, vbucket.."

      Both the buckets on source/destination are available for a long period of time, so it does not look like it is an issue w/ bucket not being ready.

      I dont have logs on this currently, will add soon.

      Seeing this across platforms - linux / windows and on most 2.0.1 runs.

      2.Replication replicates data as expected, and these error messages persist for over an hour on the xdcr-last 10 errors. This gives user a wrong idea about the state of replication.

      The intial Replication-call should either wait long enough to avoid these errors/ figure out if something else can be done here.

      And also, how frequently do we clean up the xdcr-error messages on the console? Can we clear them sooner than current time?

        Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          ketaki Ketaki Gangal added a comment -

          Please release-note this.

          Show
          ketaki Ketaki Gangal added a comment - Please release-note this.
          Show
          ketaki Ketaki Gangal added a comment - Logs from the node it is trying to reach : clusters http://ec2-54-235-229-199.compute-1.amazonaws.com:8091/index.html#sec=replications to http://ec2-107-22-40-124.compute-1.amazonaws.com:8091/index.html#sec=analytics&statsBucket=%2Fpools%2Fdefault%2Fbuckets%2Fsasl%3Fbucket_uuid%3De1f9d1e199f28b83c35f26c61ee90ec9
          Hide
          ketaki Ketaki Gangal added a comment -

          Hi Aliaksey,

          I ve added logs from one of the nodes. Could you take a look?

          Please re-assign this to me/ Jin after you do so.

          thanks,
          Ketaki

          Show
          ketaki Ketaki Gangal added a comment - Hi Aliaksey, I ve added logs from one of the nodes. Could you take a look? Please re-assign this to me/ Jin after you do so. thanks, Ketaki
          Hide
          ketaki Ketaki Gangal added a comment -

          Change added here http://review.couchbase.org/#/c/24986/, will be part of next branch.

          Show
          ketaki Ketaki Gangal added a comment - Change added here http://review.couchbase.org/#/c/24986/ , will be part of next branch.
          Show
          ketaki Ketaki Gangal added a comment - http://review.couchbase.org/#/c/24986/
          Hide
          kzeller kzeller added a comment -

          Added as known issue to RN 2.0.1:

          When you create a replication between two clusters, you
          may experience the incorrect error message
          "Failed to grab remote bucket info, vbucket". Replication will start as
          and function expected, but the incorrect error message may persist for some time.
          Please ignore this incorrect error.

          Show
          kzeller kzeller added a comment - Added as known issue to RN 2.0.1: When you create a replication between two clusters, you may experience the incorrect error message "Failed to grab remote bucket info, vbucket". Replication will start as and function expected, but the incorrect error message may persist for some time. Please ignore this incorrect error.
          Hide
          Aliaksey Artamonau Aliaksey Artamonau added a comment -

          I would not call the error incorrect. It's just that replication is able to recover from it.

          Show
          Aliaksey Artamonau Aliaksey Artamonau added a comment - I would not call the error incorrect. It's just that replication is able to recover from it.
          Hide
          kzeller kzeller added a comment -

          Redo as:

          When you create a replication between two clusters, you
          may experience two error messages:
          "Failed to grab remote bucket info, vbucket" and "Error replicating vbucket X". Nonetheless,
          replication
          will still start and then function as expected, but the error messages may appear
          for some time in the Web Console. Please ignore this behavior.

          Show
          kzeller kzeller added a comment - Redo as: When you create a replication between two clusters, you may experience two error messages: "Failed to grab remote bucket info, vbucket" and "Error replicating vbucket X". Nonetheless, replication will still start and then function as expected, but the error messages may appear for some time in the Web Console. Please ignore this behavior.
          Hide
          Aliaksey Artamonau Aliaksey Artamonau added a comment -

          Looks good to me.

          Show
          Aliaksey Artamonau Aliaksey Artamonau added a comment - Looks good to me.
          Hide
          kzeller kzeller added a comment -

          Yes indeed.... : )

          Show
          kzeller kzeller added a comment - Yes indeed.... : )
          Show
          kzeller kzeller added a comment - http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-server-rn_2-0-0l.html
          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          Not sure who to assign on.

          Code-wise we've fixed it. It was caused by thundering herd of those remote bucket info requests and we don't allow that anymore.

          I believe folks wanted to add this to release note.

          Anyways Aliaksey is done with that.

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - Not sure who to assign on. Code-wise we've fixed it. It was caused by thundering herd of those remote bucket info requests and we don't allow that anymore. I believe folks wanted to add this to release note. Anyways Aliaksey is done with that.
          Hide
          maria Maria McDuff (Inactive) added a comment -

          karen, are you finished documenting this?
          just flagging this for you for release note.

          Will assign to Ketaki for verification/closing.

          Thanks.

          Show
          maria Maria McDuff (Inactive) added a comment - karen, are you finished documenting this? just flagging this for you for release note. Will assign to Ketaki for verification/closing. Thanks.
          Hide
          kzeller kzeller added a comment -

          I add this to the 2.0.1 release notes as minor known issue to ignore. Is the message now fixed for 2.0.2?

          Show
          kzeller kzeller added a comment - I add this to the 2.0.1 release notes as minor known issue to ignore. Is the message now fixed for 2.0.2?
          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          As can be seen above it is fixed.

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - As can be seen above it is fixed.
          Hide
          maria Maria McDuff (Inactive) added a comment -

          pls verify / close.
          if issue is fixed, karen does not need to RN for 2.0.2

          Show
          maria Maria McDuff (Inactive) added a comment - pls verify / close. if issue is fixed, karen does not need to RN for 2.0.2
          Hide
          kzeller kzeller added a comment - - edited

          Relabeled in RN 2.0.2 as Fix. For earlier versions was in release notes as known issue.

          Show
          kzeller kzeller added a comment - - edited Relabeled in RN 2.0.2 as Fix. For earlier versions was in release notes as known issue.

            People

            • Assignee:
              ketaki Ketaki Gangal
              Reporter:
              ketaki Ketaki Gangal
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes