Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 4.0.0
-
Fix Version/s: 5.0.0
-
Component/s: tools
-
Labels:None
-
Environment:Ubuntu 14.04 under AWS, when deploying with Juju.
-
Triage:Untriaged
-
Operating System:Ubuntu 64-bit
-
Link to Log File, atop/blg, CBCollectInfo, Core dump:CBCollectInfo attached.
-
Is this a Regression?:No
Description
I get two couchbase servers ready. Server A and server B. Server A is the master node, which was installed first. Server B was installed later and is instructed to join the cluster, by issuing the 'rebalance' command. However, this happens:
First try:
INFO cluster-relation-changed + /opt/couchbase/bin/couchbase-cli rebalance -u juju -p charmtest -c 172.31.23.65 --server-add=172.31.5.101 --server-add-username=juju --server-add-password=charmtest --service=data,index,query
|
INFO cluster-relation-changed ERROR: unable to server-add 172.31.5.101:8091 (400) Bad Request
|
INFO cluster-relation-changed ["Prepare join failed. Could not connect to \"172.31.5.101\" on port 8091. This could be due to an incorrect host/port combination or a firewall in place between the servers."]
|
ERROR juju.worker.uniter.operation runhook.go:107 hook "cluster-relation-changed" failed: exit status 2
|
|
|
|
|
It goes like this for a couple times. Then, this happens (here's the weird part):
|
|
|
INFO cluster-relation-changed + /opt/couchbase/bin/couchbase-cli rebalance -u juju -p charmtest -c 172.31.23.65 --server-add=172.31.5.101 --server-add-username=juju --server-add-password=charmtest --service=data,index,query
|
INFO cluster-relation-changed INFO: rebalancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
|
INFO cluster-relation-changed SUCCESS: rebalanced cluster
|
INFO cluster-relation-changed close failed in file object destructor:
|
INFO cluster-relation-changed sys.excepthook is missing
|
INFO cluster-relation-changed lost sys.stderr
|
I don't know why this error is coming up. After that, any future attempts would get this:
INFO cluster-relation-changed ERROR: unable to server-add 172.31.5.101:8091 (400) Bad Request
|
INFO cluster-relation-changed ["Prepare join failed. Node is already part of cluster."]
|
So, couchbase is in fact rebalancing the cluster with the node. However, for some weird reason, it's failing. All of these logs are from server B.