Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45900

[CX] /pools/default/nodeServicesStreaming provides no indication of self rebalance out before closing

    XMLWordPrintable

Details

    • Task
    • Resolution: Cannot Reproduce
    • Critical
    • 7.0.0
    • Cheshire-Cat
    • analytics
    • 1

    Description

      In 6.6.x, the pools/default/nodeServicesStreaming endpoint on a node being rebalanced out provides an update indicating that the node is in fact getting rebalanced out prior to closing. In Cheshire-Cat, the connection simply closes.

      As analytics uses the output of this endpoint to maintain information on remote link topology changes, we are unaware of the rebalance out.

      e.g. in 6.6.2 when n_0 is being rebalanced out:

      $ curl -u couchbase:couchbase http://localhost:9001/pools/default/nodeServicesStreaming
      {"rev":13,"nodesExt":[{"services":{"mgmt":9001,"mgmtSSL":19001,"kv":12002,"kvSSL":11994,"capi":9501,"capiSSL":19501,"projector":10001,"projector":10001},"thisNode":true,"hostname":"192.168.33.218"}],"clusterCapabilitiesVer":[1,0],"clusterCapabilities":{"n1ql":["enhancedPreparedStatements"]}}
       
      ....
      {"rev":2032,"nodesExt":[{"services":{"mgmt":9001,"mgmtSSL":19001,"kv":12002,"kvSSL":11994,"capi":9501,"capiSSL":19501,"projector":10001,"projector":10001},"thisNode":true,"hostname":"192.168.33.218","alternateAddresses":{"external":{"hostname":"0.0.0.0","ports":{"mgmt":9001,"mgmtSSL":19001,"kv":12002,"kvSSL":11994,"capi":9501,"capiSSL":19501,"projector":10001,"projector":10001}}}},{"services":{"mgmt":9002,"mgmtSSL":19002,"kv":12004,"kvSSL":11990,"capi":9502,"capiSSL":19502,"projector":10002,"projector":10002},"hostname":"192.168.33.218","alternateAddresses":{"external":{"hostname":"0.0.0.0","ports":{"mgmt":9002,"mgmtSSL":19002,"kv":12004,"kvSSL":11990,"capi":9502,"capiSSL":19502,"projector":10002,"projector":10002}}}}],"clusterCapabilitiesVer":[1,0],"clusterCapabilities":{"n1ql":["enhancedPreparedStatements"]}}
       
      {"rev":3017,"nodesExt":[{"services":{"mgmt":9002,"mgmtSSL":19002,"kv":12004,"kvSSL":11990,"capi":9502,"capiSSL":19502,"projector":10002,"projector":10002},"hostname":"192.168.33.218","alternateAddresses":{"external":{"hostname":"0.0.0.0","ports":{"mgmt":9002,"mgmtSSL":19002,"kv":12004,"kvSSL":11990,"capi":9502,"capiSSL":19502,"projector":10002,"projector":10002}}}}],"clusterCapabilitiesVer":[1,0],"clusterCapabilities":{"n1ql":["enhancedPreparedStatements"]}}
       
       
      curl: (18) transfer closed with outstanding read data remaining
      
      

      vs 7.0, the last nodeServices update still includes n_0:

      $ curl -u couchbase:couchbase http://localhost:9001/pools/default/nodeServicesStreaming
      {"rev":22,"nodesExt":[{"services":{"capi":9501,"capiSSL":19501,"kv":12002,"kvSSL":11994,"mgmt":9001,"mgmtSSL":19001,"projector":10001},"thisNode":true,"hostname":"192.168.1.236"}],"clusterCapabilitiesVer":[1,0],"clusterCapabilities":{"n1ql":["enhancedPreparedStatements"]}}
       
      ...
       
      {"rev":141,"nodesExt":[{"services":{"capi":9501,"capiSSL":19501,"kv":12002,"kvSSL":11994,"mgmt":9001,"mgmtSSL":19001,"projector":10001},"thisNode":true,"hostname":"192.168.1.236","alternateAddresses":{"external":{"hostname":"0.0.0.0","ports":{"mgmt":9001,"mgmtSSL":19001,"kv":12002,"kvSSL":11994,"capi":9501,"capiSSL":19501,"projector":10001,"projector":10001}}}},{"services":{"capi":9502,"capiSSL":19502,"kv":12004,"kvSSL":11990,"mgmt":9002,"mgmtSSL":19002,"projector":10002},"hostname":"192.168.1.236","alternateAddresses":{"external":{"hostname":"0.0.0.0","ports":{"mgmt":9002,"mgmtSSL":19002,"kv":12004,"kvSSL":11990,"capi":9502,"capiSSL":19502,"projector":10002,"projector":10002}}}}],"clusterCapabilitiesVer":[1,0],"clusterCapabilities":{"n1ql":["enhancedPreparedStatements"]}}
       
       
      curl: (18) transfer closed with outstanding read data remaining
      
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              michael.blow Michael Blow
              michael.blow Michael Blow
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty