/controller/addNode throws error 400, "Unexpected server error, request logged

[2015-05-08 16:45:53,738] - [task:369] INFO - adding node 172.23.121.163:8091 to cluster

[2015-05-08 16:45:53,738] - [rest_client:948] INFO - adding remote node @172.23.121.163:8091 to this cluster @172.23.121.159:8091

[2015-05-08 16:45:53,791] - [rest_client:731] ERROR - http://172.23.121.159:8091/controller/addNode error 400 reason: unknown ["Prepare join failed. Got HTTP status 500 from REST call post to http://172.23.121.163:8091/engageCluster2. Body was: \"[\\\"Unexpected server error, request logged.\\\"]\""]

[2015-05-08 16:45:53,811] - [rest_client:2186] INFO - Latest logs from UI on 172.23.121.159:

[2015-05-08 16:45:53,811] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 5, u'text': u'Failed to add node 172.23.121.163:8091 to cluster. Prepare join failed. Got HTTP status 500 from REST call post to http://172.23.121.163:8091/engageCluster2. Body was: "[\\"Unexpected server error, request logged.\\"]"', u'shortText': u'message', u'serverTime': u'2015-05-08T16:45:45.384Z', u'module': u'ns_cluster', u'tstamp': 1431128745384, u'type': u'info'}

[2015-05-08 16:45:53,812] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.160', u'code': 0, u'text': u'Bucket "default" loaded on node \'ns_1@172.23.121.160\' in 0 seconds.', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:26.600Z', u'module': u'ns_memcached', u'tstamp': 1431128366600, u'type': u'info'}

[2015-05-08 16:45:53,812] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 0, u'text': u'Bucket "default" loaded on node \'ns_1@172.23.121.159\' in 0 seconds.', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:26.124Z', u'module': u'ns_memcached', u'tstamp': 1431128366124, u'type': u'info'}

[2015-05-08 16:45:53,813] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 12, u'text': u'Created bucket "default" of type: couchbase\n[{num_replicas,1},\n {replica_index,true},\n {ram_quota,1605369856},\n {auth_type,sasl},\n {flush_enabled,true},\n {num_threads,3},\n {eviction_policy,value_only}]', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:26.071Z', u'module': u'menelaus_web', u'tstamp': 1431128366071, u'type': u'info'}

[2015-05-08 16:45:53,813] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 0, u'text': u'Reset auto-failover count', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:12.040Z', u'module': u'auto_failover', u'tstamp': 1431128352040, u'type': u'info'}

[2015-05-08 16:45:53,813] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 1, u'text': u'Rebalance completed successfully.\n', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:12.039Z', u'module': u'ns_orchestrator', u'tstamp': 1431128352039, u'type': u'info'}

[2015-05-08 16:45:53,814] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 4, u'text': u"Starting rebalance, KeepNodes = ['ns_1@172.23.121.159','ns_1@172.23.121.160'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes\n", u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:12.006Z', u'module': u'ns_orchestrator', u'tstamp': 1431128352006, u'type': u'info'}

[2015-05-08 16:45:53,814] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.160', u'code': 3, u'text': u'Node ns_1@172.23.121.160 joined cluster', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:11.905Z', u'module': u'ns_cluster', u'tstamp': 1431128351905, u'type': u'info'}

[2015-05-08 16:45:53,814] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.160', u'code': 1, u'text': u'Couchbase Server has started on web port 8091 on node \'ns_1@172.23.121.160\'. Version: "4.0.0-2046-rel-enterprise".', u'shortText': u'web start ok', u'serverTime': u'2015-05-08T16:39:11.904Z', u'module': u'menelaus_sup', u'tstamp': 1431128351904, u'type': u'info'}

[2015-05-08 16:45:53,815] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 4, u'text': u"Node 'ns_1@172.23.121.159' saw that node 'ns_1@172.23.121.160' came up. Tags: []", u'shortText': u'node up', u'serverTime': u'2015-05-08T16:39:07.862Z', u'module': u'ns_node_disco', u'tstamp': 1431128347862, u'type': u'info'}

[2015-05-08 16:45:53,823] - [rest_client:2186] INFO - Latest logs from UI on 172.23.121.163:

[2015-05-08 16:45:53,824] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 5, u'text': u'Failed to add node 172.23.121.163:8091 to cluster. Prepare join failed. Got HTTP status 500 from REST call post to http://172.23.121.163:8091/engageCluster2. Body was: "[\\"Unexpected server error, request logged.\\"]"', u'shortText': u'message', u'serverTime': u'2015-05-08T16:45:45.384Z', u'module': u'ns_cluster', u'tstamp': 1431128745384, u'type': u'info'}

[2015-05-08 16:45:53,824] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.160', u'code': 0, u'text': u'Bucket "default" loaded on node \'ns_1@172.23.121.160\' in 0 seconds.', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:26.600Z', u'module': u'ns_memcached', u'tstamp': 1431128366600, u'type': u'info'}

[2015-05-08 16:45:53,825] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 0, u'text': u'Bucket "default" loaded on node \'ns_1@172.23.121.159\' in 0 seconds.', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:26.124Z', u'module': u'ns_memcached', u'tstamp': 1431128366124, u'type': u'info'}

[2015-05-08 16:45:53,825] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 12, u'text': u'Created bucket "default" of type: couchbase\n[{num_replicas,1},\n {replica_index,true},\n {ram_quota,1605369856},\n {auth_type,sasl},\n {flush_enabled,true},\n {num_threads,3},\n {eviction_policy,value_only}]', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:26.071Z', u'module': u'menelaus_web', u'tstamp': 1431128366071, u'type': u'info'}

[2015-05-08 16:45:53,825] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 0, u'text': u'Reset auto-failover count', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:12.040Z', u'module': u'auto_failover', u'tstamp': 1431128352040, u'type': u'info'}

[2015-05-08 16:45:53,826] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 1, u'text': u'Rebalance completed successfully.\n', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:12.039Z', u'module': u'ns_orchestrator', u'tstamp': 1431128352039, u'type': u'info'}

[2015-05-08 16:45:53,826] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 4, u'text': u"Starting rebalance, KeepNodes = ['ns_1@172.23.121.159','ns_1@172.23.121.160'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes\n", u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:12.006Z', u'module': u'ns_orchestrator', u'tstamp': 1431128352006, u'type': u'info'}

[2015-05-08 16:45:53,827] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.160', u'code': 3, u'text': u'Node ns_1@172.23.121.160 joined cluster', u'shortText': u'message', u'serverTime': u'2015-05-08T16:39:11.905Z', u'module': u'ns_cluster', u'tstamp': 1431128351905, u'type': u'info'}

[2015-05-08 16:45:53,827] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.160', u'code': 1, u'text': u'Couchbase Server has started on web port 8091 on node \'ns_1@172.23.121.160\'. Version: "4.0.0-2046-rel-enterprise".', u'shortText': u'web start ok', u'serverTime': u'2015-05-08T16:39:11.904Z', u'module': u'menelaus_sup', u'tstamp': 1431128351904, u'type': u'info'}

[2015-05-08 16:45:53,827] - [rest_client:2187] ERROR - {u'node': u'ns_1@172.23.121.159', u'code': 4, u'text': u"Node 'ns_1@172.23.121.159' saw that node 'ns_1@172.23.121.160' came up. Tags: []", u'shortText': u'node up', u'serverTime': u'2015-05-08T16:39:07.862Z', u'module': u'ns_node_disco', u'tstamp': 1431128347862, u'type': u'info'}

[2015-05-08 16:45:53,828] - [rest_client:992] ERROR - add_node error : ["Prepare join failed. Got HTTP status 500 from REST call post to http://172.23.121.163:8091/engageCluster2. Body was: \"[\\\"Unexpected server error, request logged.\\\"]\""]

ERROR