Details
-
Bug
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
Untriaged
-
Centos 64-bit
-
1
-
Yes
Description
Failing on 7.0.0-2477. Last good build : 7.0.0-2417
Steps:
1. Setup C1 and C2 clusters
C1 : 172.23.123.65, 172.23.123.66
C2 : 172.23.123.42, 172.23.123.44
2. Bidirectional replication between C1 and C2
C1->C2 Replication id: 7db79a5455988422dd445909507d36bb/default/default
C2->C1 Replication id: 7d46b625f361ab329b75cc78d622a87e/default/default
3. Swap-rebalance [remove_node:172.23.123.44] -> [add_node:172.23.122.97] in C2
4. After swap rebalance is complete, the rest call to get list of buckets on C1 fails:
2020-07-01 11:21:07 | ERROR | MainProcess | test_thread | [rest_client._http_request] GET http://172.23.123.65:8091/pools/default/buckets?basic_stats=true body: headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 500 reason: unknown b'["Unexpected server error, request logged."]' auth: Administrator:password |
Around the same time, I see this is goxdcr.log on C1:
2020-07-01T11:21:01.297-07:00 INFO GOXDCR.ThrSeqTrackSvc: 7db79a5455988422dd445909507d36bb/default/default_ThroughSeqnoTracker time_spent=383.405µs num_vb=512 max_sent=0 avg_sent=0 max_filtered=0 avg_filtered=0 max_failed_cr=0 avg_failed_cr=0 max_gap=0 avg_gap=0 |
2020-07-01T11:21:01.321-07:00 ERRO GOXDCR.ClusterInfoSvc: Failed to unmarshal the response as json, err=json: cannot unmarshal array into Go value of type map[string]interface {}, bod=["Unexpected server error, request logged."] |
res=&{500 Internal Server Error 500 HTTP/1.1 1 1 map[Server:[Couchbase Server] X-Content-Type-Options:[nosniff] X-Frame-Options:[DENY] Cache-Control:[no-cache,no-store,must-revalidate] Content-Length:[44] Content-Type:[application/json] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Pragma:[no-cache] Date:[Wed, 01 Jul 2020 18:21:00 GMT] X-Permitted-Cross-Domain-Policies:[none] X-Xss-Protection:[1; mode=block]] 0xc0027db1c0 44 [] false false map[] 0xc0001d6600 <nil>} |
2020-07-01T11:21:01.321-07:00 ERRO GOXDCR.ClusterInfoSvc: Failed to get bucket info for bucket 'default'. host=127.0.0.1:8091, err=json: cannot unmarshal array into Go value of type map[string]interface {}, statusCode=500 |
2020-07-01T11:21:01.321-07:00 INFO GOXDCR.TopoChangeDet: TopologyChangeDetectorSvc for pipeline 7db79a5455988422dd445909507d36bb/default/default validateSourceTopology completed |
2020-07-01T11:21:01.321-07:00 WARN GOXDCR.TopoChangeDet: TopologyChangeDetectorSvc for pipeline 7db79a5455988422dd445909507d36bb/default/default received error when validating source topology change. err=Failed to get bucket info. |
2020-07-01T11:21:01.321-07:00 INFO GOXDCR.TopoChangeDet: TopologyChangeDetectorSvc for pipeline 7db79a5455988422dd445909507d36bb/default/default handleSourceTopologyChange completed |
2020-07-01T11:21:01.328-07:00 INFO GOXDCR.TopoChangeDet: TopologyChangeDetectorSvc for pipeline 7db79a5455988422dd445909507d36bb/default/default validateTargetTopology completed |
2020-07-01T11:21:01.328-07:00 WARN GOXDCR.TopoChangeDet: TopologyChangeDetectorSvc for pipeline 7db79a5455988422dd445909507d36bb/default/default received error when validating target topology change. err=Topology has changed on target cluster |
This is the test cmd:
./testrunner -i /tmp/testexec.20569.ini -p stop-on-failure=False,fail_on_errors=1,get-cbcollect-info=False,get-cbcollect-info=True -t xdcr.rebalanceXDCR.Rebalance.swap_rebalance_replication_with_ddoc_compaction,items=50000,rdirection=bidirection,is_dev_ddoc=false,rebalance=C2,GROUP=P2,poll_timeout=900,stop-on-failure=False,fail_on_errors=1,get-cbcollect-info=False |
Attachments
Issue Links
- relates to
-
-
- Closed
-