Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-43828

Node takes a minute after swap-out rebalance to become ready

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown
    • CX Sprint 234

    Description

      I'm doing upgrade testing from MH to CC and noticed that after a rebalance out of the MH node the remaining CC node doesn't transition from orange to green in the UI for a minute. I ran a number of iterations and it seems to involved CBAS. If I do the experiment with only KV the orange->green occurs a couple of seconds after rebalance completes. If I then add CBAS to the config the orange->green occurs about a minute after rebalance completes.

      I've gathered a cbcollect
      https://s3.amazonaws.com/cb-engineering/stevewatanabe-22JAN2021/collectinfo-2021-01-22T183919-n_1%40127.0.0.1.zip

      Looking at the analytics_debug.log file I see these at the start

      2021-01-22T10:29:00.232-08:00 INFO CBAS.bootstrap.AnalyticsNCApplication [main] Starting Couchbase Analytics driver, version: {"build.date":"Thu Aug 06 20:47:27 UTC 2020","build.number":"7909","build.version":"6.6.0-7909"}
      2021-01-22T10:29:00.824-08:00 DEBU CBAS.bootstrap.Auditor [Executor-9:758a53da704ee3772760159f888dfc4d] updated audit settings update: AuditSettings{disabled=[8255, 20485, 20488, 20489, 20490, 20491, 20493, 28672, 28673, 28674, 28675, 28676, 28677, 28678, 28679, 28680, 28681, 28682, 28683, 28684, 28685, 28686, 28687, 28688, 28689, 28690, 28691, 28692, 28693, 28694, 28695, 28697, 28698, 28699, 28700, 28701, 28702, 28704, 28705, 32770, 32771, 32772, 32780, 32783, 32784, 32785, 32786, 36867, 36868, 36869, 36870, 36871, 36872, 36873, 36877, 36878, 36879, 40963], uid='8228582', auditdEnabled=false, disabledUsers=<ud>[]</ud>}
      2021-01-22T10:29:00.854-08:00 DEBU CBAS.util.MXHelper [main] ignoring exception calling RuntimeMXBean.getBootClassPath; returning null
      java.lang.UnsupportedOperationException: Boot class path mechanism is not supported
          at sun.management.RuntimeImpl.getBootClassPath(Unknown Source) ~[?:?]
          at org.apache.hyracks.util.MXHelper.getBootClassPath(MXHelper.java:111) [hyracks-util.jar:6.6.0-7909]
          at org.apache.hyracks.control.common.controllers.NodeRegistration.<init>(NodeRegistration.java:110) [hyracks-control-common.jar:6.6.0-7909]
          at org.apache.hyracks.control.nc.NodeControllerService.initNodeControllerState(NodeControllerService.java:334) [hyracks-control-nc.jar:6.6.0-7909]
          at org.apache.hyracks.control.nc.NodeControllerService.start(NodeControllerService.java:299) [hyracks-control-nc.jar:6.6.0-7909]
          at com.couchbase.analytics.control.AnalyticsDriver.startService(AnalyticsDriver.java:128) [cbas-server.jar:6.6.0-7909]
          at com.couchbase.analytics.control.AnalyticsDriver.main(AnalyticsDriver.java:99) [cbas-server.jar:6.6.0-7909]
      

      and then many java backtraces in the file.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              steve.watanabe Steve Watanabe
              steve.watanabe Steve Watanabe
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty