Uploaded image for project: 'Couchbase Go SDK'
  1. Couchbase Go SDK
  2. GOCBC-1219

Transaction fails after rebalancing out KV node

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • core-10.0.7
    • None
    • None
    • 7.1.0-2021
    • 1

    Description

      To repro:

      • setup 3 node cluster:
        • node 1: kv, index, query
        • node 2: kv
        • node 3: kv
      • load travel-sample
      • Run query in transaction: SELECT * FROM `travel-sample` LIMIT 100;
      • Remove node 2 and rebalance
      • Run query in transaction: SELECT * FROM `travel-sample` LIMIT 100;

      Get following error post rebalance:

      cbq> select * from `travel-sample` limit 100;
      {
          "requestID": "ebc0a81d-f981-4314-87ad-78746a061645",
          "signature": {
              "*": "*"
          },
          "results": [
          {
              "travel-sample": {
                  "callsign": "TXW",
                  "country": "United States",
                  "iata": "TQ",
                  "icao": "TXW",
                  "id": 10123,
                  "name": "Texas Wings",
                  "type": "airline"
              }
          },
          {
              "travel-sample": {
                  "callsign": "atifly",
                  "country": "United States",
                  "iata": "A1",
                  "icao": "A1F",
                  "id": 10226,
                  "name": "Atifly",
                  "type": "airline"
              }
          },
          {
              "travel-sample": {
                  "callsign": null,
                  "country": "United Kingdom",
                  "iata": null,
                  "icao": "JRB",
                  "id": 10642,
                  "name": "Jc royal.britannica",
                  "type": "airline"
              }
          },
          {
              "travel-sample": {
                  "callsign": "LOCAIR",
                  "country": "United States",
                  "iata": "ZQ",
                  "icao": "LOC",
                  "id": 10748,
                  "name": "Locair",
                  "type": "airline"
              }
          }
          ],
          "errors": [
              {
                  "cause": {
                      "cause": {
                          "-": {
                              "InnerError": {
                                  "InnerError": {},
                                  "Message": "unambiguous timeout"
                              }
                          },
                          "i": "0x0",
                          "s": "LookupIn",
                          "t": 2500388
                      },
                      "raise": "failed",
                      "retry": true,
                      "rollback": true
                  },
                  "code": 17017,
                  "msg": "Transaction fetch error"
              }
          ],
          "status": "errors",
          "metrics": {
              "elapsedTime": "2.512617007s",
              "executionTime": "2.512372351s",
              "resultCount": 4,
              "resultSize": 1036,
              "serviceLoad": 1,
              "transactionElapsedTime": "6.43192018s",
              "transactionRemainingTime": "1m53.567929584s",
              "errorCount": 1
          }
      }
       
       

      in query.log:

      2022-01-14T11:00:24.568-08:00 [WARN] (TXGOCBCORE) Failed to connect to host. specific server is invalid
      2022-01-14T11:00:26.380-08:00 [WARN] (TXGOCBCORE) Pipeline Client 0xc00190a660 failed to bootstrap: authentication failure | {"status_code":36,"document_key":"travel-sample","bucket":"travel-sample","error_name":"EACCESS","error_description":"Not authorized for command","opaque":3,"last_dispatched_to":"172.23.104.91:11210","last_dispatched_from":"172.23.104.90:37836","last_connection_id":"e7d8a655ba0a2241/20a95918987402ea"}
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pierre.regazzoni Pierre Regazzoni
              pierre.regazzoni Pierre Regazzoni
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty