Details
-
Bug
-
Resolution: Not a Bug
-
Critical
-
Cheshire-Cat
-
6.6.2-9588 -> 7.0.0-5226
-
Untriaged
-
Centos 64-bit
-
1
-
Unknown
Description
Steps to Repro
1. Run the following longevity on 6.6.2 for 3-4 days
./sequoia -client 172.23.96.162:2375 -provider file:centos_third_cluster.yml -test tests/integration/test_allFeatures_madhatter_durability.yml -scope tests/integration/scope_Xattrs_Madhatter.yml -scale 3 -repeat 0 -log_level 0 -version 6.6.2-9588 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=604800 -show_topology=true
|
2. We have 27 node cluster in 6.6.2
3. Add 6 nodes(1 of each service - 7.0.0-5226) and remove 6 nodes(6.6.2) and do a swap rebalance to upgrade the cluster.
4. Failover 6 node(1 of each service - 6.6.2), upgrade, do a recovery and rebalance. Noticed errors like the following.
172.23.106.117 : query
Sample result for error message 'Error getting documents for infer.
|
Error getting random entry from keyspace - cause: MCResponse status=KEY_ENOENT, opcode=0xb6, opaque=0, msg: ' at time 2021-05-25T09:29:59.691-07:00: [{u'completed_requests': {u'node': u'172.23.106.117:8091', u'errors': [{u'message': u'Error getting documents for infer.\nError getting random entry from keyspace - cause: MCResponse status=KEY_ENOENT, opcode=0xb6, opaque=0, msg: ', u'code': 0, u'key': u''}], u'scanConsistency': u'unbounded', u'state': u'completed', u'phaseOperators': {u'authorize': 1}, u'serviceTime': u'1.282805054s', u'remoteAddr': u'172.23.104.244:43354', u'elapsedTime': u'1.282884582s', u'resultSize': 0, u'requestTime': u'2021-05-25T09:29:59.691-07:00', u'statement': u'infer `CUSTOMER` with {"infer_timeout":5, "max_schema_MB":1};', u'requestId': u'd274208a-664e-4f55-a7ce-8ca36c1e34ee', u'clientContextID': u'INTERNAL-0cd5e617-d9f5-43ab-b87c-c4e0f900a537', u'userAgent': u'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.
|
0.4430.212 Safari/537.36 (Couchbase Query Workbench (6.6.2-9588-enterprise))', u'users': u'Administrator', u'resultCount': 0, u'errorCount': 1, u'phaseTimes': {u'authorize': u'40.249382ms', u'parse': u'410.08\xb5s', u'instantiate': u'14.985\xb5s', u'run': u'1.282333589s', u'plan': u'21.389\xb5s'}}}]
|
This was not seen on upgrade from 6.6.2-9588 -> 7.0.0-5141. cbcollect_info attached.