2020/12/21 09:09:48.374231 dev test MDW TST ConnectionAndKeyValueCircuitBre --------------------ConnectionAndKeyValueCircuitBreakers_ChangePassword-------------------- 2020/12/21 09:09:48.374575 dev test MDW DBG ForkExec About to exec: [./scripts/start_mock.sh] [ --harakiri-monitor=localhost:45321 --buckets=mock:testpwd:couchbase --nodes=4 --replicas=2] 2020/12/21 09:09:48.637978 dev test MDW INFO CouchbaseMock Received admin port 45009 2020/12/21 09:09:48.638031 dev test MDW TST MOCK Mock started successfully (attempt 1). 2020/12/21 09:09:48.638066 dev test MDW DBG CouchbaseMock Sending OOB query {"command":"GET_MCPORTS","payload":{"bucket":"mock"}} 2020/12/21 09:09:48.653911 dev test MDW DBG CouchbaseMock Received OOB reply {"payload":[50863,52179,50263,52511],"status":"ok"} 2020/12/21 09:09:48.665912 dev test MDW DBG TST Set up circuit breaker for key-value operations 2020/12/21 09:09:48.665930 dev test MDW DBG TST Set up circuit breaker for connection 2020/12/21 09:09:48.666130 dev test MDW NOT Monitoring Collector created 2020/12/21 09:09:48.666322 dev test MDW DBG TST Connecting cluster Id '' with index node1 2020/12/21 09:09:48.666334 dev test MDW INFO TST Connecting to couchbase, attempt number [ 1 / 2 ] 2020/12/21 09:09:48.671745 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:48.671761 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:48.671765 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:48.671775 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:48.671782 dev test MDW INFO TST Connecting to server node1:8091 2020/12/21 09:09:48.671955 dev test MDW INFO TST Connecting to server couchbase://node1/adeltour-br3?bootstrap_on=cccp&dnssrv=off&compression=off on bucket adeltour-br3 with user [ adeltour-br3 ] 2020/12/21 09:09:48.672032 dev test MDW INFO instance Version=3.0.7, Changeset=d2d3666e269b41413408aaae96eb289f1cb4f8ec 2020/12/21 09:09:48.672041 dev test MDW INFO instance Effective connection string: couchbase://node1/adeltour-br3?bootstrap_on=cccp&dnssrv=off&compression=off. Bucket=adeltour-br3 2020/12/21 09:09:48.672087 dev test MDW DBG instance Applying initial cntl compression=off 2020/12/21 09:09:48.672106 dev test MDW DBG instance Adding host node1:8091 to initial HTTP bootstrap list 2020/12/21 09:09:48.672129 dev test MDW DBG instance Adding host node1:11210 to initial CCCP bootstrap list 2020/12/21 09:09:48.672137 dev test MDW DBG instance Bootstrap hosts loaded (cccp:1, http:1) 2020/12/21 09:09:48.672168 dev test MDW INFO bootstrap Requested network configuration: heuristic 2020/12/21 09:09:48.672176 dev test MDW DBG confmon Preparing providers (this may be called multiple times) 2020/12/21 09:09:48.672179 dev test MDW DBG confmon Provider CCCP is ENABLED 2020/12/21 09:09:48.672184 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:48.672209 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:48.672213 dev test MDW INFO cccp Requesting connection to node node1:11210 for CCCP configuration 2020/12/21 09:09:48.672225 dev test MDW DBG lcbio_mgr (HE=0x5ef91b0) Creating new connection because none are available in the pool 2020/12/21 09:09:48.672234 dev test MDW DBG lcbio_mgr (HE=0x5ef91b0) New pool entry: I=0x5efbf40 2020/12/21 09:09:48.672249 dev test MDW INFO connection (SOCK=899e70ccb8404fc5) Starting. Timeout=2000000us 2020/12/21 09:09:48.672265 dev test MDW DBG connection (SOCK=899e70ccb8404fc5) Created new socket with FD=16 2020/12/21 09:09:48.672301 dev test MDW DBG connection (SOCK=899e70ccb8404fc5) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:48.673126 dev test MDW INFO connection (SOCK=899e70ccb8404fc5) Connected established 2020/12/21 09:09:48.673139 dev test MDW DBG connection (SOCK=899e70ccb8404fc5) Successfully set TCP_NODELAY 2020/12/21 09:09:48.673147 dev test MDW DBG connection (SOCK=899e70ccb8404fc5) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:48.673151 dev test MDW DBG lcbio_mgr (HE=0x5ef91b0) Received result for I=0x5efbf40,C=(nil); E=0x0 2020/12/21 09:09:48.673155 dev test MDW DBG lcbio_mgr (HE=0x5ef91b0) Assigning R=0x5efbe50 SOCKET=0x5efc120 2020/12/21 09:09:48.673174 dev test MDW DBG ioctx (CTX=0x5efce80,unknown) Pairing with SOCK=899e70ccb8404fc5 2020/12/21 09:09:48.673237 dev test MDW DBG negotiation (CTX=0x5efce80,sasl,SASLREQ=0x5efcc00) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"b846923ecf2d2b84/899e70ccb8404fc5"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x12 (Collections), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:48.674309 dev test MDW DBG negotiation (CTX=0x5efce80,sasl,SASLREQ=0x5efcc00) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:48.675007 dev test MDW DBG negotiation (CTX=0x5efce80,sasl,SASLREQ=0x5efcc00) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:48.681436 dev test MDW WARN negotiation (CTX=0x5efce80,sasl,SASLREQ=0x5efcc00) SASL auth failed with STATUS=0x20 2020/12/21 09:09:48.681468 dev test MDW ERROR [0/] negotiation (CTX=0x5efce80,sasl,SASLREQ=0x5efcc00) Error: 0xce, SASL Step failed (ref: "3b7c2b41-2ee5-460f-e9e8-686ebb877e74") 2020/12/21 09:09:48.681478 dev test MDW ERROR [0/] cccp (CTX=(nil),) Could not get configuration: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:48.681481 dev test MDW INFO confmon Provider 'CCCP' failed: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:48.681483 dev test MDW WARN confmon Received authentication error during bootstrap 2020/12/21 09:09:48.681486 dev test MDW DBG confmon Maximum provider reached. Resetting index 2020/12/21 09:09:48.681492 dev test MDW ERROR [0/] bootstrap Failed to bootstrap client=0x5ef9430. Error=LCB_ERR_NO_MATCHING_SERVER (1010) (Last=LCB_ERR_AUTHENTICATION_FAILURE (206)), Message=No more bootstrap providers remain 2020/12/21 09:09:48.681502 dev test MDW INFO Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Error during connection sequence 2020/12/21 09:09:48.681510 dev test MDW DBG ioctx (CTX=0x5efce80,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:48.681599 dev test MDW INFO TST Retrying connection with another password. 2020/12/21 09:09:48.681639 dev test MDW INFO TST Connecting to server couchbase://node1/adeltour-br3?bootstrap_on=cccp&dnssrv=off&compression=off on bucket adeltour-br3 with user [ adeltour-br3 ] 2020/12/21 09:09:48.681663 dev test MDW INFO instance Version=3.0.7, Changeset=d2d3666e269b41413408aaae96eb289f1cb4f8ec 2020/12/21 09:09:48.681671 dev test MDW INFO instance Effective connection string: couchbase://node1/adeltour-br3?bootstrap_on=cccp&dnssrv=off&compression=off. Bucket=adeltour-br3 2020/12/21 09:09:48.681681 dev test MDW DBG instance Applying initial cntl compression=off 2020/12/21 09:09:48.681687 dev test MDW DBG instance Adding host node1:8091 to initial HTTP bootstrap list 2020/12/21 09:09:48.681698 dev test MDW DBG instance Adding host node1:11210 to initial CCCP bootstrap list 2020/12/21 09:09:48.681709 dev test MDW DBG instance Bootstrap hosts loaded (cccp:1, http:1) 2020/12/21 09:09:48.681721 dev test MDW INFO bootstrap Requested network configuration: heuristic 2020/12/21 09:09:48.681725 dev test MDW DBG confmon Preparing providers (this may be called multiple times) 2020/12/21 09:09:48.681727 dev test MDW DBG confmon Provider CCCP is ENABLED 2020/12/21 09:09:48.681729 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:48.681734 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:48.681737 dev test MDW INFO cccp Requesting connection to node node1:11210 for CCCP configuration 2020/12/21 09:09:48.681742 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) Creating new connection because none are available in the pool 2020/12/21 09:09:48.681749 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) New pool entry: I=0x5ef8480 2020/12/21 09:09:48.681755 dev test MDW INFO connection (SOCK=af5f258011eac731) Starting. Timeout=2000000us 2020/12/21 09:09:48.681769 dev test MDW DBG connection (SOCK=af5f258011eac731) Created new socket with FD=16 2020/12/21 09:09:48.681792 dev test MDW DBG connection (SOCK=af5f258011eac731) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:48.682454 dev test MDW INFO connection (SOCK=af5f258011eac731) Connected established 2020/12/21 09:09:48.682465 dev test MDW DBG connection (SOCK=af5f258011eac731) Successfully set TCP_NODELAY 2020/12/21 09:09:48.682468 dev test MDW DBG connection (SOCK=af5f258011eac731) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:48.682470 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) Received result for I=0x5ef8480,C=(nil); E=0x0 2020/12/21 09:09:48.682473 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) Assigning R=0x5f17360 SOCKET=0x5f16550 2020/12/21 09:09:48.682478 dev test MDW DBG ioctx (CTX=0x5f189a0,unknown) Pairing with SOCK=af5f258011eac731 2020/12/21 09:09:48.682505 dev test MDW DBG negotiation (CTX=0x5f189a0,sasl,SASLREQ=0x5f187b0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/af5f258011eac731"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x12 (Collections), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:48.683381 dev test MDW DBG negotiation (CTX=0x5f189a0,sasl,SASLREQ=0x5f187b0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:48.683967 dev test MDW DBG negotiation (CTX=0x5f189a0,sasl,SASLREQ=0x5f187b0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:48.689805 dev test MDW DBG negotiation (CTX=0x5f189a0,sasl,SASLREQ=0x5f187b0) Sending SELECT_BUCKET "adeltour-br3" 2020/12/21 09:09:48.690309 dev test MDW DBG ioctx (CTX=0x5f189a0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:48.690321 dev test MDW DBG ioctx (CTX=0x5f16920,unknown) Pairing with SOCK=af5f258011eac731 2020/12/21 09:09:48.690876 dev test MDW DBG ioctx (CTX=0x5f16920,bc_cccp) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:48.690887 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) Placing socket back into the pool. I=0x5ef8480,C=0x5f16550 2020/12/21 09:09:48.691366 dev test MDW INFO confmon Setting initial configuration. Received via CCCP (bucket="adeltour-br3", rev=52291, address="node1") 2020/12/21 09:09:48.691387 dev test MDW DBG cccp Node 3 has no data service 2020/12/21 09:09:48.691392 dev test MDW DBG cccp Node 4 has no data service 2020/12/21 09:09:48.691394 dev test MDW DBG cccp Node 5 has no data service 2020/12/21 09:09:48.691423 dev test MDW INFO bootstrap Selected network configuration: "default" 2020/12/21 09:09:48.691432 dev test MDW INFO Connection sequence successful 2020/12/21 09:09:48.691445 dev test MDW DBG Success Notifying success 2020/12/21 09:09:48.691462 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:48.691471 dev test MDW DBG Success Notifying success 2020/12/21 09:09:48.691474 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:48.691476 dev test MDW INFO TST Succeed to connect to server node1 2020/12/21 09:09:48.691482 dev test MDW INFO TST Connected to Couchbase: node1 2020/12/21 09:09:48.691492 dev test MDW INFO KV Set active cluster with index '0' (1 clusters instanciated) 2020/12/21 09:09:48.691517 dev test MDW INFO TST Registered handler for variable TST_COUCHBASE_CONNECTION_ENDPOINTS 2020/12/21 09:09:48.691542 dev test MDW TST TST CurlHttpClient::CurlHttpClient start 2020/12/21 09:09:48.691572 dev test MDW TST TST CurlHttpClient::CurlHttpClient end 2020/12/21 09:09:48.691665 dev test MDW TST TST Libcurl: STATE: INIT => CONNECT handle 0x5f11638; line 1657 (connection #-5000) 2020/12/21 09:09:48.691716 dev test MDW TST TST Libcurl: Added connection 0. The cache now contains 1 members 2020/12/21 09:09:48.691734 dev test MDW TST TST Libcurl: family0 == v4, family1 == v6 2020/12/21 09:09:48.691751 dev test MDW TST TST Libcurl: Trying node1:8091... 2020/12/21 09:09:48.691787 dev test MDW TST TST Libcurl: STATE: CONNECT => WAITCONNECT handle 0x5f11638; line 1718 (connection #0) 2020/12/21 09:09:48.692636 dev test MDW TST TST Libcurl: Connected to node1 (node1) port 8091 (#0) 2020/12/21 09:09:48.692647 dev test MDW TST TST Libcurl: STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x5f11638; line 1851 (connection #0) 2020/12/21 09:09:48.692650 dev test MDW TST TST Libcurl: Marked for [keep alive]: HTTP default 2020/12/21 09:09:48.692653 dev test MDW TST TST Libcurl: STATE: SENDPROTOCONNECT => DO handle 0x5f11638; line 1872 (connection #0) 2020/12/21 09:09:48.692667 dev test MDW TST TST Libcurl: Server auth using Basic with user 'adeltour-br3' 2020/12/21 09:09:48.692700 dev test MDW TST TST Libcurl: POST /controller/changePassword HTTP/1.1 Host: node1:8091 Authorization: Basic YWRlbHRvdXItYnIzOnRlc3Rwd2Q= User-Agent: kvclient_unittests Accept: */* Content-Type: application/x-www-form-urlencoded Content-Length: 22 Expect: 100-continue 2020/12/21 09:09:48.692712 dev test MDW TST TST Libcurl: STATE: DO => DO_DONE handle 0x5f11638; line 1943 (connection #0) 2020/12/21 09:09:48.692717 dev test MDW TST TST Libcurl: STATE: DO_DONE => PERFORM handle 0x5f11638; line 2064 (connection #0) 2020/12/21 09:09:48.694615 dev test MDW TST TST Libcurl: Mark bundle as not supporting multiuse 2020/12/21 09:09:48.694628 dev test MDW TST TST Libcurl: HTTP 1.1 or later with persistent connection 2020/12/21 09:09:48.694631 dev test MDW TST TST Libcurl: HTTP/1.1 100 Continue 2020/12/21 09:09:48.694679 dev test MDW TST TST Libcurl: password=wrongpassword 2020/12/21 09:09:48.694688 dev test MDW TST TST Libcurl: We are completely uploaded and fine 2020/12/21 09:09:48.710473 dev test MDW TST TST Libcurl: Mark bundle as not supporting multiuse 2020/12/21 09:09:48.710482 dev test MDW TST TST Libcurl: HTTP 1.1 or later with persistent connection 2020/12/21 09:09:48.710484 dev test MDW TST TST Libcurl: HTTP/1.1 200 OK 2020/12/21 09:09:48.710495 dev test MDW TST TST Libcurl: X-XSS-Protection: 1; mode=block 2020/12/21 09:09:48.710506 dev test MDW TST TST Libcurl: X-Permitted-Cross-Domain-Policies: none 2020/12/21 09:09:48.710519 dev test MDW TST TST Libcurl: X-Frame-Options: DENY 2020/12/21 09:09:48.710530 dev test MDW TST TST Libcurl: X-Content-Type-Options: nosniff 2020/12/21 09:09:48.710541 dev test MDW TST TST Libcurl: Server: Couchbase Server 2020/12/21 09:09:48.710552 dev test MDW TST TST Libcurl: Pragma: no-cache 2020/12/21 09:09:48.710565 dev test MDW TST TST Libcurl: Expires: Thu, 01 Jan 1970 00:00:00 GMT 2020/12/21 09:09:48.710577 dev test MDW TST TST Libcurl: Date: Mon, 21 Dec 2020 09:09:48 GMT 2020/12/21 09:09:48.710595 dev test MDW TST TST Libcurl: Content-Length: 0 2020/12/21 09:09:48.710605 dev test MDW TST TST Libcurl: Cache-Control: no-cache,no-store,must-revalidate 2020/12/21 09:09:48.710625 dev test MDW TST TST Libcurl: 2020/12/21 09:09:48.710635 dev test MDW TST TST Libcurl: STATE: PERFORM => DONE handle 0x5f11638; line 2254 (connection #0) 2020/12/21 09:09:48.710637 dev test MDW TST TST Libcurl: multi_done 2020/12/21 09:09:48.710644 dev test MDW TST TST Libcurl: Connection #0 to host node1 left intact 2020/12/21 09:09:48.710648 dev test MDW TST TST Libcurl: Expire cleared (transfer 0x5f11638) 2020/12/21 09:09:48.710658 dev test MDW TST TST CurlHttpClient::performRequest - No error 2020/12/21 09:09:48.710660 dev test MDW TST TST CurlHttpClient::performRequest - Got Response 2020/12/21 09:09:49.714485 dev test MDW NOT TST Topology change detected. Reset all the blacklist and/or circuit breaker structures 2020/12/21 09:09:49.714593 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714610 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714618 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714623 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714626 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714635 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714637 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714640 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714643 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714647 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714650 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714652 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714658 dev test MDW DBG TST Circuit breakers after topology change: [node1:11210 - Closed] [node2:11210 - Closed] [node3:11210 - Closed] [node4:9101 - Closed] [node5:9101 - Closed] [node7:8093 - Closed] 2020/12/21 09:09:49.714673 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714676 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714680 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.714682 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.714725 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.714744 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f13780 2020/12/21 09:09:49.714752 dev test MDW INFO connection (SOCK=24211bf3cc4962f8) Starting. Timeout=100000us 2020/12/21 09:09:49.714775 dev test MDW DBG connection (SOCK=24211bf3cc4962f8) Created new socket with FD=17 2020/12/21 09:09:49.714844 dev test MDW DBG connection (SOCK=24211bf3cc4962f8) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.715602 dev test MDW INFO connection (SOCK=24211bf3cc4962f8) Connected established 2020/12/21 09:09:49.715613 dev test MDW DBG connection (SOCK=24211bf3cc4962f8) Successfully set TCP_NODELAY 2020/12/21 09:09:49.715618 dev test MDW DBG connection (SOCK=24211bf3cc4962f8) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.715621 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f13780,C=(nil); E=0x0 2020/12/21 09:09:49.715625 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f11710 SOCKET=0x5f13960 2020/12/21 09:09:49.715629 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.715637 dev test MDW DBG ioctx (CTX=0x5f0ac70,unknown) Pairing with SOCK=24211bf3cc4962f8 2020/12/21 09:09:49.715687 dev test MDW DBG negotiation (CTX=0x5f0ac70,sasl,SASLREQ=0x5f14410) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/24211bf3cc4962f8"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.716706 dev test MDW DBG negotiation (CTX=0x5f0ac70,sasl,SASLREQ=0x5f14410) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.717259 dev test MDW DBG negotiation (CTX=0x5f0ac70,sasl,SASLREQ=0x5f14410) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.723127 dev test MDW WARN negotiation (CTX=0x5f0ac70,sasl,SASLREQ=0x5f14410) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.723157 dev test MDW ERROR [0/] negotiation (CTX=0x5f0ac70,sasl,SASLREQ=0x5f14410) Error: 0xce, SASL Step failed (ref: "e3ca0166-98cd-45d9-1be9-f680eed45bc1") 2020/12/21 09:09:49.723169 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.723182 dev test MDW DBG retryq Adding PKT=0x5f0c950 to retry queue. retries=1, opaque=0, time=8486us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.723185 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.723192 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.723199 dev test MDW DBG ioctx (CTX=0x5f0ac70,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.723231 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.723242 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:49.723265 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) Found ready connection in pool. Reusing socket and not creating new connection 2020/12/21 09:09:49.723276 dev test MDW DBG lcbio_mgr (HE=0x5f115c0) Assigning R=0x5f11750 SOCKET=0x5f16550 2020/12/21 09:09:49.723280 dev test MDW DBG server (SRV=0x5f15800) Got new KV connection (json=yes, snappy=no, mt=yes, durability=no, bucket=yes "adeltour-br3") 2020/12/21 09:09:49.723283 dev test MDW DBG ioctx (CTX=0x5f0ca20,unknown) Pairing with SOCK=af5f258011eac731 2020/12/21 09:09:49.724391 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.734476 dev test MDW DBG retryq Flush PKT=0x5f0c950 to network. retries=1, opaque=0, IX=2, time=19777us 2020/12/21 09:09:49.734489 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.734496 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c820 2020/12/21 09:09:49.734501 dev test MDW INFO connection (SOCK=471cf43928da8635) Starting. Timeout=100000us 2020/12/21 09:09:49.734511 dev test MDW DBG connection (SOCK=471cf43928da8635) Created new socket with FD=17 2020/12/21 09:09:49.734537 dev test MDW DBG connection (SOCK=471cf43928da8635) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.735241 dev test MDW INFO connection (SOCK=471cf43928da8635) Connected established 2020/12/21 09:09:49.735252 dev test MDW DBG connection (SOCK=471cf43928da8635) Successfully set TCP_NODELAY 2020/12/21 09:09:49.735255 dev test MDW DBG connection (SOCK=471cf43928da8635) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.735258 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c820,C=(nil); E=0x0 2020/12/21 09:09:49.735261 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f0c6f0 SOCKET=0x5f42c60 2020/12/21 09:09:49.735264 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.735269 dev test MDW DBG ioctx (CTX=0x5f0d000,unknown) Pairing with SOCK=471cf43928da8635 2020/12/21 09:09:49.735295 dev test MDW DBG negotiation (CTX=0x5f0d000,sasl,SASLREQ=0x5f42d10) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/471cf43928da8635"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.736254 dev test MDW DBG negotiation (CTX=0x5f0d000,sasl,SASLREQ=0x5f42d10) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.736794 dev test MDW DBG negotiation (CTX=0x5f0d000,sasl,SASLREQ=0x5f42d10) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.742683 dev test MDW WARN negotiation (CTX=0x5f0d000,sasl,SASLREQ=0x5f42d10) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.742711 dev test MDW ERROR [0/] negotiation (CTX=0x5f0d000,sasl,SASLREQ=0x5f42d10) Error: 0xce, SASL Step failed (ref: "51e980b9-6ae3-40bc-dfba-a57602e603f8") 2020/12/21 09:09:49.742721 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.742731 dev test MDW DBG retryq Adding PKT=0x5f23dc0 to retry queue. retries=2, opaque=0, time=28034us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.742733 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:49.742736 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.742744 dev test MDW DBG ioctx (CTX=0x5f0d000,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.742779 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.742788 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.742791 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:49.742805 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.742810 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f0c700 2020/12/21 09:09:49.742815 dev test MDW INFO connection (SOCK=9e02371aca1fa46e) Starting. Timeout=100000us 2020/12/21 09:09:49.742825 dev test MDW DBG connection (SOCK=9e02371aca1fa46e) Created new socket with FD=17 2020/12/21 09:09:49.742855 dev test MDW DBG connection (SOCK=9e02371aca1fa46e) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.743682 dev test MDW INFO connection (SOCK=9e02371aca1fa46e) Connected established 2020/12/21 09:09:49.743692 dev test MDW DBG connection (SOCK=9e02371aca1fa46e) Successfully set TCP_NODELAY 2020/12/21 09:09:49.743696 dev test MDW DBG connection (SOCK=9e02371aca1fa46e) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.743699 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f0c700,C=(nil); E=0x0 2020/12/21 09:09:49.743703 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f42c70 SOCKET=0x5f0c940 2020/12/21 09:09:49.743706 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.743710 dev test MDW DBG ioctx (CTX=0x5f0d1d0,unknown) Pairing with SOCK=9e02371aca1fa46e 2020/12/21 09:09:49.743735 dev test MDW DBG negotiation (CTX=0x5f0d1d0,sasl,SASLREQ=0x5f0cf80) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/9e02371aca1fa46e"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.744667 dev test MDW DBG negotiation (CTX=0x5f0d1d0,sasl,SASLREQ=0x5f0cf80) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.745216 dev test MDW DBG negotiation (CTX=0x5f0d1d0,sasl,SASLREQ=0x5f0cf80) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.751091 dev test MDW WARN negotiation (CTX=0x5f0d1d0,sasl,SASLREQ=0x5f0cf80) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.751119 dev test MDW ERROR [0/] negotiation (CTX=0x5f0d1d0,sasl,SASLREQ=0x5f0cf80) Error: 0xce, SASL Step failed (ref: "5f1cf4c5-d2f0-49ba-2dcb-4bb17755bc45") 2020/12/21 09:09:49.751129 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.751134 dev test MDW DBG retryq Adding PKT=0x5f241e0 to retry queue. retries=1, opaque=3, time=8336us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.751136 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.751141 dev test MDW DBG retryq Adding PKT=0x5f242e0 to retry queue. retries=1, opaque=4, time=8341us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.751143 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.751150 dev test MDW DBG ioctx (CTX=0x5f0d1d0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.761237 dev test MDW DBG retryq Flush PKT=0x5f241e0 to network. retries=1, opaque=3, IX=0, time=18437us 2020/12/21 09:09:49.761254 dev test MDW DBG retryq Flush PKT=0x5f242e0 to network. retries=1, opaque=4, IX=0, time=18434us 2020/12/21 09:09:49.761256 dev test MDW DBG retryq Flush PKT=0x5f23dc0 to network. retries=2, opaque=0, IX=2, time=46539us 2020/12/21 09:09:49.761262 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.761279 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f31f60 2020/12/21 09:09:49.761284 dev test MDW INFO connection (SOCK=388de95c1f929b49) Starting. Timeout=100000us 2020/12/21 09:09:49.761295 dev test MDW DBG connection (SOCK=388de95c1f929b49) Created new socket with FD=17 2020/12/21 09:09:49.761319 dev test MDW DBG connection (SOCK=388de95c1f929b49) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.762362 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.762390 dev test MDW INFO connection (SOCK=388de95c1f929b49) Connected established 2020/12/21 09:09:49.762399 dev test MDW DBG connection (SOCK=388de95c1f929b49) Successfully set TCP_NODELAY 2020/12/21 09:09:49.762402 dev test MDW DBG connection (SOCK=388de95c1f929b49) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.762405 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f31f60,C=(nil); E=0x0 2020/12/21 09:09:49.762409 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f42dd0 SOCKET=0x5f0c1b0 2020/12/21 09:09:49.762411 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.762416 dev test MDW DBG ioctx (CTX=0x5f0cec0,unknown) Pairing with SOCK=388de95c1f929b49 2020/12/21 09:09:49.762442 dev test MDW DBG negotiation (CTX=0x5f0cec0,sasl,SASLREQ=0x5f0c700) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/388de95c1f929b49"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.763348 dev test MDW DBG negotiation (CTX=0x5f0cec0,sasl,SASLREQ=0x5f0c700) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.763892 dev test MDW DBG negotiation (CTX=0x5f0cec0,sasl,SASLREQ=0x5f0c700) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.769660 dev test MDW WARN negotiation (CTX=0x5f0cec0,sasl,SASLREQ=0x5f0c700) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.769688 dev test MDW ERROR [0/] negotiation (CTX=0x5f0cec0,sasl,SASLREQ=0x5f0c700) Error: 0xce, SASL Step failed (ref: "2e1d19c9-21fa-4863-a1b9-645130a47e72") 2020/12/21 09:09:49.769698 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.769708 dev test MDW DBG retryq Adding PKT=0x5f32330 to retry queue. retries=3, opaque=0, time=55013us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.769710 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:49.769714 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.769720 dev test MDW DBG ioctx (CTX=0x5f0cec0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.769755 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.769763 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.769766 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:49.769780 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.769787 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c700 2020/12/21 09:09:49.769795 dev test MDW INFO connection (SOCK=22389a72ffc7569d) Starting. Timeout=100000us 2020/12/21 09:09:49.769805 dev test MDW DBG connection (SOCK=22389a72ffc7569d) Created new socket with FD=17 2020/12/21 09:09:49.769827 dev test MDW DBG connection (SOCK=22389a72ffc7569d) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.770370 dev test MDW INFO connection (SOCK=22389a72ffc7569d) Connected established 2020/12/21 09:09:49.770380 dev test MDW DBG connection (SOCK=22389a72ffc7569d) Successfully set TCP_NODELAY 2020/12/21 09:09:49.770383 dev test MDW DBG connection (SOCK=22389a72ffc7569d) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.770385 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c700,C=(nil); E=0x0 2020/12/21 09:09:49.770388 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f42de0 SOCKET=0x5f0c910 2020/12/21 09:09:49.770390 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.770395 dev test MDW DBG ioctx (CTX=0x5f32250,unknown) Pairing with SOCK=22389a72ffc7569d 2020/12/21 09:09:49.770421 dev test MDW DBG negotiation (CTX=0x5f32250,sasl,SASLREQ=0x5f32030) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/22389a72ffc7569d"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.771203 dev test MDW DBG negotiation (CTX=0x5f32250,sasl,SASLREQ=0x5f32030) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.771770 dev test MDW DBG negotiation (CTX=0x5f32250,sasl,SASLREQ=0x5f32030) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.777236 dev test MDW WARN negotiation (CTX=0x5f32250,sasl,SASLREQ=0x5f32030) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.777265 dev test MDW ERROR [0/] negotiation (CTX=0x5f32250,sasl,SASLREQ=0x5f32030) Error: 0xce, SASL Step failed (ref: "9567afd8-5f0c-4c0d-2f26-ab269388004a") 2020/12/21 09:09:49.777275 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.777285 dev test MDW DBG retryq Adding PKT=0x5f24310 to retry queue. retries=1, opaque=5, time=7509us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.777287 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.777291 dev test MDW DBG retryq Adding PKT=0x5f24020 to retry queue. retries=1, opaque=6, time=7514us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.777293 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.777301 dev test MDW DBG ioctx (CTX=0x5f32250,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.787394 dev test MDW DBG retryq Flush PKT=0x5f24310 to network. retries=1, opaque=5, IX=0, time=17616us 2020/12/21 09:09:49.787407 dev test MDW DBG retryq Flush PKT=0x5f24020 to network. retries=1, opaque=6, IX=0, time=17614us 2020/12/21 09:09:49.787409 dev test MDW DBG retryq Next tick in 12 ms 2020/12/21 09:09:49.788722 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.800804 dev test MDW DBG retryq Flush PKT=0x5f32330 to network. retries=3, opaque=0, IX=2, time=86107us 2020/12/21 09:09:49.800825 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.800831 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f42c70 2020/12/21 09:09:49.800836 dev test MDW INFO connection (SOCK=23339615ea039b32) Starting. Timeout=100000us 2020/12/21 09:09:49.800849 dev test MDW DBG connection (SOCK=23339615ea039b32) Created new socket with FD=17 2020/12/21 09:09:49.800873 dev test MDW DBG connection (SOCK=23339615ea039b32) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.801614 dev test MDW INFO connection (SOCK=23339615ea039b32) Connected established 2020/12/21 09:09:49.801624 dev test MDW DBG connection (SOCK=23339615ea039b32) Successfully set TCP_NODELAY 2020/12/21 09:09:49.801627 dev test MDW DBG connection (SOCK=23339615ea039b32) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.801639 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f42c70,C=(nil); E=0x0 2020/12/21 09:09:49.801644 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f31f70 SOCKET=0x5f240b0 2020/12/21 09:09:49.801646 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.801651 dev test MDW DBG ioctx (CTX=0x5f0c8b0,unknown) Pairing with SOCK=23339615ea039b32 2020/12/21 09:09:49.801677 dev test MDW DBG negotiation (CTX=0x5f0c8b0,sasl,SASLREQ=0x5f0c4f0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/23339615ea039b32"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.802605 dev test MDW DBG negotiation (CTX=0x5f0c8b0,sasl,SASLREQ=0x5f0c4f0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.803129 dev test MDW DBG negotiation (CTX=0x5f0c8b0,sasl,SASLREQ=0x5f0c4f0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.808880 dev test MDW WARN negotiation (CTX=0x5f0c8b0,sasl,SASLREQ=0x5f0c4f0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.808908 dev test MDW ERROR [0/] negotiation (CTX=0x5f0c8b0,sasl,SASLREQ=0x5f0c4f0) Error: 0xce, SASL Step failed (ref: "2e730e9a-9164-4f6d-fe3f-1f82a1dc9d60") 2020/12/21 09:09:49.808912 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.808918 dev test MDW DBG retryq Adding PKT=0x5f42de0 to retry queue. retries=4, opaque=0, time=94222us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.808920 dev test MDW DBG retryq Next tick in 5 ms 2020/12/21 09:09:49.808923 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.808930 dev test MDW DBG ioctx (CTX=0x5f0c8b0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.808964 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.808973 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.808976 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:49.809959 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.816042 dev test MDW WARN retryq Failing command (pkt=0x5f42de0, opaque=0, retries=4, time=101344us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.816070 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword0" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [101ms] 2020/12/21 09:09:49.816108 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:49.816126 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:49.816139 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:49.816146 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:49.816150 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:49.816155 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:49.816161 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:49.816175 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword0], on master server [node3:11210], global elapsed time: [101ms] 2020/12/21 09:09:49.816207 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.816217 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.816219 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.816221 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.833861 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword1" ; lcb error: LCB_SUCCESS (0): Success (Not an error), global elapsed time: [17ms] 2020/12/21 09:09:49.833877 dev test MDW DBG Success Notifying success 2020/12/21 09:09:49.833883 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:49.833885 dev test MDW DBG Success Notifying success 2020/12/21 09:09:49.833888 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:49.833900 dev test MDW DBG TST Stored "ConnectionAndKeyValueCircuitBreakers_ChangePassword1" CAS: 1608541789563191296 on host node1:8091 2020/12/21 09:09:49.834060 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.834071 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.834075 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.834077 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.834092 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.834099 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f5dec0 2020/12/21 09:09:49.834104 dev test MDW INFO connection (SOCK=0e60c4843e3d3550) Starting. Timeout=100000us 2020/12/21 09:09:49.834113 dev test MDW DBG connection (SOCK=0e60c4843e3d3550) Created new socket with FD=17 2020/12/21 09:09:49.834142 dev test MDW DBG connection (SOCK=0e60c4843e3d3550) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.834893 dev test MDW INFO connection (SOCK=0e60c4843e3d3550) Connected established 2020/12/21 09:09:49.834903 dev test MDW DBG connection (SOCK=0e60c4843e3d3550) Successfully set TCP_NODELAY 2020/12/21 09:09:49.834907 dev test MDW DBG connection (SOCK=0e60c4843e3d3550) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.834910 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f5dec0,C=(nil); E=0x0 2020/12/21 09:09:49.834914 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5db60 SOCKET=0x5f5dce0 2020/12/21 09:09:49.834917 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.834921 dev test MDW DBG ioctx (CTX=0x5f5ed30,unknown) Pairing with SOCK=0e60c4843e3d3550 2020/12/21 09:09:49.834948 dev test MDW DBG negotiation (CTX=0x5f5ed30,sasl,SASLREQ=0x5f5eae0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/0e60c4843e3d3550"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.835891 dev test MDW DBG negotiation (CTX=0x5f5ed30,sasl,SASLREQ=0x5f5eae0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.836470 dev test MDW DBG negotiation (CTX=0x5f5ed30,sasl,SASLREQ=0x5f5eae0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.842203 dev test MDW WARN negotiation (CTX=0x5f5ed30,sasl,SASLREQ=0x5f5eae0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.842232 dev test MDW ERROR [0/] negotiation (CTX=0x5f5ed30,sasl,SASLREQ=0x5f5eae0) Error: 0xce, SASL Step failed (ref: "d855d2d4-9e3b-4a80-121c-116ec32bb23e") 2020/12/21 09:09:49.842242 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.842249 dev test MDW DBG retryq Adding PKT=0x5f6fa40 to retry queue. retries=1, opaque=9, time=8165us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.842251 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.842254 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.842261 dev test MDW DBG ioctx (CTX=0x5f5ed30,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.842296 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.842304 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.842307 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:49.842322 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.842327 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f5df20 2020/12/21 09:09:49.842337 dev test MDW INFO connection (SOCK=00301774a2684c07) Starting. Timeout=100000us 2020/12/21 09:09:49.842347 dev test MDW DBG connection (SOCK=00301774a2684c07) Created new socket with FD=17 2020/12/21 09:09:49.842374 dev test MDW DBG connection (SOCK=00301774a2684c07) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.842933 dev test MDW INFO connection (SOCK=00301774a2684c07) Connected established 2020/12/21 09:09:49.842943 dev test MDW DBG connection (SOCK=00301774a2684c07) Successfully set TCP_NODELAY 2020/12/21 09:09:49.842947 dev test MDW DBG connection (SOCK=00301774a2684c07) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.842949 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f5df20,C=(nil); E=0x0 2020/12/21 09:09:49.842953 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5de30 SOCKET=0x5f5e100 2020/12/21 09:09:49.842955 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.842959 dev test MDW DBG ioctx (CTX=0x5f5ee00,unknown) Pairing with SOCK=00301774a2684c07 2020/12/21 09:09:49.842985 dev test MDW DBG negotiation (CTX=0x5f5ee00,sasl,SASLREQ=0x5f5ebb0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/00301774a2684c07"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.843812 dev test MDW DBG negotiation (CTX=0x5f5ee00,sasl,SASLREQ=0x5f5ebb0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.844355 dev test MDW DBG negotiation (CTX=0x5f5ee00,sasl,SASLREQ=0x5f5ebb0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.850349 dev test MDW WARN negotiation (CTX=0x5f5ee00,sasl,SASLREQ=0x5f5ebb0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.850377 dev test MDW ERROR [0/] negotiation (CTX=0x5f5ee00,sasl,SASLREQ=0x5f5ebb0) Error: 0xce, SASL Step failed (ref: "36f2b225-279b-4518-45ba-9d8996f9d06f") 2020/12/21 09:09:49.850387 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.850393 dev test MDW DBG retryq Adding PKT=0x5f5de30 to retry queue. retries=1, opaque=10, time=8075us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.850395 dev test MDW DBG retryq Next tick in 1 ms 2020/12/21 09:09:49.850399 dev test MDW DBG retryq Adding PKT=0x5f6eb70 to retry queue. retries=1, opaque=11, time=8081us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.850401 dev test MDW DBG retryq Next tick in 1 ms 2020/12/21 09:09:49.850408 dev test MDW DBG ioctx (CTX=0x5f5ee00,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.852499 dev test MDW DBG retryq Flush PKT=0x5f6fa40 to network. retries=1, opaque=9, IX=1, time=18414us 2020/12/21 09:09:49.852513 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.852518 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f5eda0 2020/12/21 09:09:49.852523 dev test MDW INFO connection (SOCK=567898130884bb58) Starting. Timeout=100000us 2020/12/21 09:09:49.852534 dev test MDW DBG connection (SOCK=567898130884bb58) Created new socket with FD=17 2020/12/21 09:09:49.852573 dev test MDW DBG connection (SOCK=567898130884bb58) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.852584 dev test MDW DBG retryq Next tick in 7 ms 2020/12/21 09:09:49.853279 dev test MDW INFO connection (SOCK=567898130884bb58) Connected established 2020/12/21 09:09:49.853290 dev test MDW DBG connection (SOCK=567898130884bb58) Successfully set TCP_NODELAY 2020/12/21 09:09:49.853293 dev test MDW DBG connection (SOCK=567898130884bb58) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.853296 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f5eda0,C=(nil); E=0x0 2020/12/21 09:09:49.853299 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5df20 SOCKET=0x5f5f070 2020/12/21 09:09:49.853302 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.853307 dev test MDW DBG ioctx (CTX=0x5f6e960,unknown) Pairing with SOCK=567898130884bb58 2020/12/21 09:09:49.853333 dev test MDW DBG negotiation (CTX=0x5f6e960,sasl,SASLREQ=0x5f6e7a0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/567898130884bb58"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.854259 dev test MDW DBG negotiation (CTX=0x5f6e960,sasl,SASLREQ=0x5f6e7a0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.854783 dev test MDW DBG negotiation (CTX=0x5f6e960,sasl,SASLREQ=0x5f6e7a0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.860540 dev test MDW WARN negotiation (CTX=0x5f6e960,sasl,SASLREQ=0x5f6e7a0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.860572 dev test MDW ERROR [0/] negotiation (CTX=0x5f6e960,sasl,SASLREQ=0x5f6e7a0) Error: 0xce, SASL Step failed (ref: "53636c70-97b7-4293-1cd8-f73cc25be225") 2020/12/21 09:09:49.860576 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.860582 dev test MDW DBG retryq Adding PKT=0x5f6f950 to retry queue. retries=2, opaque=9, time=26498us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.860584 dev test MDW DBG retryq Next tick in 0 ms 2020/12/21 09:09:49.860592 dev test MDW DBG ioctx (CTX=0x5f6e960,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.860623 dev test MDW DBG retryq Flush PKT=0x5f5de30 to network. retries=1, opaque=10, IX=0, time=18304us 2020/12/21 09:09:49.860635 dev test MDW DBG retryq Flush PKT=0x5f6eb70 to network. retries=1, opaque=11, IX=0, time=18302us 2020/12/21 09:09:49.860643 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:49.861685 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.870767 dev test MDW DBG server (CTX=(nil),,SRV=0x5f016c0,IX=2) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:49.880827 dev test MDW DBG retryq Flush PKT=0x5f6f950 to network. retries=2, opaque=9, IX=1, time=46743us 2020/12/21 09:09:49.880840 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.880846 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f6ed00 2020/12/21 09:09:49.880851 dev test MDW INFO connection (SOCK=bba1391f494c3d44) Starting. Timeout=100000us 2020/12/21 09:09:49.880861 dev test MDW DBG connection (SOCK=bba1391f494c3d44) Created new socket with FD=17 2020/12/21 09:09:49.880884 dev test MDW DBG connection (SOCK=bba1391f494c3d44) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.881559 dev test MDW INFO connection (SOCK=bba1391f494c3d44) Connected established 2020/12/21 09:09:49.881570 dev test MDW DBG connection (SOCK=bba1391f494c3d44) Successfully set TCP_NODELAY 2020/12/21 09:09:49.881574 dev test MDW DBG connection (SOCK=bba1391f494c3d44) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.881577 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f6ed00,C=(nil); E=0x0 2020/12/21 09:09:49.881580 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f6fac0 SOCKET=0x5f6ec00 2020/12/21 09:09:49.881582 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.881587 dev test MDW DBG ioctx (CTX=0x5f2e6b0,unknown) Pairing with SOCK=bba1391f494c3d44 2020/12/21 09:09:49.881613 dev test MDW DBG negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/bba1391f494c3d44"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.882694 dev test MDW DBG negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.883204 dev test MDW DBG negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.888834 dev test MDW WARN negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.888862 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) Error: 0xce, SASL Step failed (ref: "d0e5accb-c5e7-4d7c-7268-5ee874399cfd") 2020/12/21 09:09:49.888872 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.888879 dev test MDW DBG retryq Adding PKT=0x5f2ed30 to retry queue. retries=3, opaque=9, time=54796us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.888881 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:49.888885 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.888892 dev test MDW DBG ioctx (CTX=0x5f2e6b0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.888925 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.888933 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.888937 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:49.888947 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.888953 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f2e650 2020/12/21 09:09:49.888958 dev test MDW INFO connection (SOCK=f40946694dd65c9f) Starting. Timeout=100000us 2020/12/21 09:09:49.888969 dev test MDW DBG connection (SOCK=f40946694dd65c9f) Created new socket with FD=17 2020/12/21 09:09:49.888990 dev test MDW DBG connection (SOCK=f40946694dd65c9f) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.889850 dev test MDW INFO connection (SOCK=f40946694dd65c9f) Connected established 2020/12/21 09:09:49.889860 dev test MDW DBG connection (SOCK=f40946694dd65c9f) Successfully set TCP_NODELAY 2020/12/21 09:09:49.889863 dev test MDW DBG connection (SOCK=f40946694dd65c9f) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.889866 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f2e650,C=(nil); E=0x0 2020/12/21 09:09:49.889870 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f6ed00 SOCKET=0x5f2e8a0 2020/12/21 09:09:49.889872 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.889877 dev test MDW DBG ioctx (CTX=0x5f2e6b0,unknown) Pairing with SOCK=f40946694dd65c9f 2020/12/21 09:09:49.889903 dev test MDW DBG negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/f40946694dd65c9f"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.891014 dev test MDW DBG negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.891572 dev test MDW DBG negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.897584 dev test MDW WARN negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.897612 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e6b0,sasl,SASLREQ=0x5f2e460) Error: 0xce, SASL Step failed (ref: "968b020c-d3cb-47c6-d4b1-d2d33a260c86") 2020/12/21 09:09:49.897621 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.897632 dev test MDW DBG retryq Adding PKT=0x5f2ec30 to retry queue. retries=1, opaque=12, time=8689us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.897634 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.897639 dev test MDW DBG retryq Adding PKT=0x5f2f340 to retry queue. retries=1, opaque=13, time=8696us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.897640 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.897647 dev test MDW DBG ioctx (CTX=0x5f2e6b0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.907732 dev test MDW DBG retryq Flush PKT=0x5f2ec30 to network. retries=1, opaque=12, IX=0, time=18789us 2020/12/21 09:09:49.907747 dev test MDW DBG retryq Flush PKT=0x5f2f340 to network. retries=1, opaque=13, IX=0, time=18788us 2020/12/21 09:09:49.907749 dev test MDW DBG retryq Next tick in 11 ms 2020/12/21 09:09:49.908849 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.916932 dev test MDW DBG server (CTX=0x5f0ca20,memcached,SRV=0x5f15800,IX=0) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:49.919007 dev test MDW DBG retryq Flush PKT=0x5f2ed30 to network. retries=3, opaque=9, IX=1, time=84923us 2020/12/21 09:09:49.919020 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.919027 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f5dae0 2020/12/21 09:09:49.919032 dev test MDW INFO connection (SOCK=8a3841982a7dcd45) Starting. Timeout=100000us 2020/12/21 09:09:49.919041 dev test MDW DBG connection (SOCK=8a3841982a7dcd45) Created new socket with FD=17 2020/12/21 09:09:49.919069 dev test MDW DBG connection (SOCK=8a3841982a7dcd45) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.919723 dev test MDW INFO connection (SOCK=8a3841982a7dcd45) Connected established 2020/12/21 09:09:49.919733 dev test MDW DBG connection (SOCK=8a3841982a7dcd45) Successfully set TCP_NODELAY 2020/12/21 09:09:49.919737 dev test MDW DBG connection (SOCK=8a3841982a7dcd45) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.919739 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f5dae0,C=(nil); E=0x0 2020/12/21 09:09:49.919742 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5dce0 SOCKET=0x5f2f3d0 2020/12/21 09:09:49.919744 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.919749 dev test MDW DBG ioctx (CTX=0x5f2ef20,unknown) Pairing with SOCK=8a3841982a7dcd45 2020/12/21 09:09:49.919775 dev test MDW DBG negotiation (CTX=0x5f2ef20,sasl,SASLREQ=0x5fb9de0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/8a3841982a7dcd45"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.920791 dev test MDW DBG negotiation (CTX=0x5f2ef20,sasl,SASLREQ=0x5fb9de0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.921332 dev test MDW DBG negotiation (CTX=0x5f2ef20,sasl,SASLREQ=0x5fb9de0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.927051 dev test MDW WARN negotiation (CTX=0x5f2ef20,sasl,SASLREQ=0x5fb9de0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.927078 dev test MDW ERROR [0/] negotiation (CTX=0x5f2ef20,sasl,SASLREQ=0x5fb9de0) Error: 0xce, SASL Step failed (ref: "70422758-dca3-4377-923f-2d38e73c6008") 2020/12/21 09:09:49.927088 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.927095 dev test MDW DBG retryq Adding PKT=0x5f2f230 to retry queue. retries=4, opaque=9, time=93012us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.927097 dev test MDW DBG retryq Next tick in 6 ms 2020/12/21 09:09:49.927101 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.927109 dev test MDW DBG ioctx (CTX=0x5f2ef20,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.927143 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.927151 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.927154 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:49.928198 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.935280 dev test MDW WARN retryq Failing command (pkt=0x5f2f230, opaque=9, retries=4, time=101194us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.935301 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword2" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [101ms] 2020/12/21 09:09:49.935309 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:49.935315 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:49.935325 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:49.935333 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:49.935335 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:49.935342 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:49.935345 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:49.935353 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword2], on master server [node2:11210], global elapsed time: [101ms] 2020/12/21 09:09:49.935380 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.935390 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.935392 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:49.935394 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:49.935408 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.935413 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c000 2020/12/21 09:09:49.935420 dev test MDW INFO connection (SOCK=458fd02bae20f62c) Starting. Timeout=100000us 2020/12/21 09:09:49.935430 dev test MDW DBG connection (SOCK=458fd02bae20f62c) Created new socket with FD=17 2020/12/21 09:09:49.935455 dev test MDW DBG connection (SOCK=458fd02bae20f62c) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.936171 dev test MDW INFO connection (SOCK=458fd02bae20f62c) Connected established 2020/12/21 09:09:49.936182 dev test MDW DBG connection (SOCK=458fd02bae20f62c) Successfully set TCP_NODELAY 2020/12/21 09:09:49.936186 dev test MDW DBG connection (SOCK=458fd02bae20f62c) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.936189 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c000,C=(nil); E=0x0 2020/12/21 09:09:49.936192 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f5dd90 SOCKET=0x5f6fb30 2020/12/21 09:09:49.936194 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.936199 dev test MDW DBG ioctx (CTX=0x5f2f3f0,unknown) Pairing with SOCK=458fd02bae20f62c 2020/12/21 09:09:49.936225 dev test MDW DBG negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/458fd02bae20f62c"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.937249 dev test MDW DBG negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.937813 dev test MDW DBG negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.943044 dev test MDW DBG server (CTX=(nil),,SRV=0x5f0def0,IX=1) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:49.943656 dev test MDW WARN negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.943683 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) Error: 0xce, SASL Step failed (ref: "f493a0ae-2cbe-45a1-f976-4ed79c32d190") 2020/12/21 09:09:49.943692 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.943699 dev test MDW DBG retryq Adding PKT=0x5f0bf70 to retry queue. retries=1, opaque=15, time=8299us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.943701 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.943704 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.943712 dev test MDW DBG ioctx (CTX=0x5f2f3f0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.943746 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.943754 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.943757 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:49.943767 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:49.943773 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f2f390 2020/12/21 09:09:49.943780 dev test MDW INFO connection (SOCK=fe78322e22a6822b) Starting. Timeout=100000us 2020/12/21 09:09:49.943790 dev test MDW DBG connection (SOCK=fe78322e22a6822b) Created new socket with FD=17 2020/12/21 09:09:49.943811 dev test MDW DBG connection (SOCK=fe78322e22a6822b) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.944525 dev test MDW INFO connection (SOCK=fe78322e22a6822b) Connected established 2020/12/21 09:09:49.944535 dev test MDW DBG connection (SOCK=fe78322e22a6822b) Successfully set TCP_NODELAY 2020/12/21 09:09:49.944539 dev test MDW DBG connection (SOCK=fe78322e22a6822b) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.944541 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f2f390,C=(nil); E=0x0 2020/12/21 09:09:49.944546 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f0c000 SOCKET=0x5f2e640 2020/12/21 09:09:49.944548 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.944553 dev test MDW DBG ioctx (CTX=0x5f2f3f0,unknown) Pairing with SOCK=fe78322e22a6822b 2020/12/21 09:09:49.944584 dev test MDW DBG negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/fe78322e22a6822b"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.945463 dev test MDW DBG negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.946006 dev test MDW DBG negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.951515 dev test MDW WARN negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.951543 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f3f0,sasl,SASLREQ=0x5f2f230) Error: 0xce, SASL Step failed (ref: "61174d84-ceb9-4b05-2ca5-dd70c9477f4c") 2020/12/21 09:09:49.951553 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.951558 dev test MDW DBG retryq Adding PKT=0x5f2ec10 to retry queue. retries=1, opaque=16, time=7795us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.951560 dev test MDW DBG retryq Next tick in 2 ms 2020/12/21 09:09:49.951565 dev test MDW DBG retryq Adding PKT=0x5f2edd0 to retry queue. retries=1, opaque=17, time=7801us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.951566 dev test MDW DBG retryq Next tick in 2 ms 2020/12/21 09:09:49.951574 dev test MDW DBG ioctx (CTX=0x5f2f3f0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.954661 dev test MDW DBG retryq Flush PKT=0x5f0bf70 to network. retries=1, opaque=15, IX=2, time=19259us 2020/12/21 09:09:49.954675 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.954680 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c090 2020/12/21 09:09:49.954685 dev test MDW INFO connection (SOCK=ccc39bc1b43645a0) Starting. Timeout=100000us 2020/12/21 09:09:49.954695 dev test MDW DBG connection (SOCK=ccc39bc1b43645a0) Created new socket with FD=17 2020/12/21 09:09:49.954716 dev test MDW DBG connection (SOCK=ccc39bc1b43645a0) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.954727 dev test MDW DBG retryq Next tick in 6 ms 2020/12/21 09:09:49.955319 dev test MDW INFO connection (SOCK=ccc39bc1b43645a0) Connected established 2020/12/21 09:09:49.955329 dev test MDW DBG connection (SOCK=ccc39bc1b43645a0) Successfully set TCP_NODELAY 2020/12/21 09:09:49.955333 dev test MDW DBG connection (SOCK=ccc39bc1b43645a0) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.955335 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c090,C=(nil); E=0x0 2020/12/21 09:09:49.955338 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f0d360 SOCKET=0x5f5db60 2020/12/21 09:09:49.955341 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.955346 dev test MDW DBG ioctx (CTX=0x5f2f230,unknown) Pairing with SOCK=ccc39bc1b43645a0 2020/12/21 09:09:49.955372 dev test MDW DBG negotiation (CTX=0x5f2f230,sasl,SASLREQ=0x5f2eed0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/ccc39bc1b43645a0"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.956220 dev test MDW DBG negotiation (CTX=0x5f2f230,sasl,SASLREQ=0x5f2eed0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.956763 dev test MDW DBG negotiation (CTX=0x5f2f230,sasl,SASLREQ=0x5f2eed0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.961687 dev test MDW DBG retryq Flush PKT=0x5f2ec10 to network. retries=1, opaque=16, IX=0, time=17921us 2020/12/21 09:09:49.961699 dev test MDW DBG retryq Flush PKT=0x5f2edd0 to network. retries=1, opaque=17, IX=0, time=17919us 2020/12/21 09:09:49.962262 dev test MDW WARN negotiation (CTX=0x5f2f230,sasl,SASLREQ=0x5f2eed0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.962290 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f230,sasl,SASLREQ=0x5f2eed0) Error: 0xce, SASL Step failed (ref: "90ca479a-42e7-490d-1579-8f285e22aff7") 2020/12/21 09:09:49.962300 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.962307 dev test MDW DBG retryq Adding PKT=0x5f2e930 to retry queue. retries=2, opaque=15, time=26907us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.962309 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:49.962317 dev test MDW DBG ioctx (CTX=0x5f2f230,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.962755 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.982835 dev test MDW DBG retryq Flush PKT=0x5f2e930 to network. retries=2, opaque=15, IX=2, time=47433us 2020/12/21 09:09:49.982851 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.982857 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5fb9eb0 2020/12/21 09:09:49.982867 dev test MDW INFO connection (SOCK=50aac7c1b458b419) Starting. Timeout=100000us 2020/12/21 09:09:49.982876 dev test MDW DBG connection (SOCK=50aac7c1b458b419) Created new socket with FD=17 2020/12/21 09:09:49.982900 dev test MDW DBG connection (SOCK=50aac7c1b458b419) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.983517 dev test MDW INFO connection (SOCK=50aac7c1b458b419) Connected established 2020/12/21 09:09:49.983528 dev test MDW DBG connection (SOCK=50aac7c1b458b419) Successfully set TCP_NODELAY 2020/12/21 09:09:49.983531 dev test MDW DBG connection (SOCK=50aac7c1b458b419) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.983534 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5fb9eb0,C=(nil); E=0x0 2020/12/21 09:09:49.983538 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f0d360 SOCKET=0x5f2ee60 2020/12/21 09:09:49.983540 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.983546 dev test MDW DBG ioctx (CTX=0x5f2eb80,unknown) Pairing with SOCK=50aac7c1b458b419 2020/12/21 09:09:49.983572 dev test MDW DBG negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f0bf70) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/50aac7c1b458b419"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.984419 dev test MDW DBG negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f0bf70) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.984931 dev test MDW DBG negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f0bf70) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.990437 dev test MDW WARN negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f0bf70) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.990473 dev test MDW ERROR [0/] negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f0bf70) Error: 0xce, SASL Step failed (ref: "987384d3-40bb-4f6c-b022-ad83b402938c") 2020/12/21 09:09:49.990484 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.990490 dev test MDW DBG retryq Adding PKT=0x5f5daa0 to retry queue. retries=3, opaque=15, time=55090us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.990492 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:49.990496 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:49.990503 dev test MDW DBG ioctx (CTX=0x5f2eb80,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:49.990535 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:49.990543 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:49.990546 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:49.990556 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:49.990562 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f2eb20 2020/12/21 09:09:49.990567 dev test MDW INFO connection (SOCK=7962a78b41c1fb2d) Starting. Timeout=100000us 2020/12/21 09:09:49.990577 dev test MDW DBG connection (SOCK=7962a78b41c1fb2d) Created new socket with FD=17 2020/12/21 09:09:49.990603 dev test MDW DBG connection (SOCK=7962a78b41c1fb2d) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:49.991406 dev test MDW INFO connection (SOCK=7962a78b41c1fb2d) Connected established 2020/12/21 09:09:49.991417 dev test MDW DBG connection (SOCK=7962a78b41c1fb2d) Successfully set TCP_NODELAY 2020/12/21 09:09:49.991421 dev test MDW DBG connection (SOCK=7962a78b41c1fb2d) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:49.991423 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f2eb20,C=(nil); E=0x0 2020/12/21 09:09:49.991426 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5fb9eb0 SOCKET=0x5f5db60 2020/12/21 09:09:49.991430 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:49.991435 dev test MDW DBG ioctx (CTX=0x5f2e760,unknown) Pairing with SOCK=7962a78b41c1fb2d 2020/12/21 09:09:49.991460 dev test MDW DBG negotiation (CTX=0x5f2e760,sasl,SASLREQ=0x5f0bf70) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/7962a78b41c1fb2d"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:49.992750 dev test MDW DBG negotiation (CTX=0x5f2e760,sasl,SASLREQ=0x5f0bf70) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:49.993221 dev test MDW DBG negotiation (CTX=0x5f2e760,sasl,SASLREQ=0x5f0bf70) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:49.999441 dev test MDW WARN negotiation (CTX=0x5f2e760,sasl,SASLREQ=0x5f0bf70) SASL auth failed with STATUS=0x20 2020/12/21 09:09:49.999469 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e760,sasl,SASLREQ=0x5f0bf70) Error: 0xce, SASL Step failed (ref: "d878caed-5232-4c6f-cc1e-8be197906308") 2020/12/21 09:09:49.999479 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:49.999484 dev test MDW DBG retryq Adding PKT=0x5f2f1a0 to retry queue. retries=1, opaque=18, time=8932us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.999486 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.999491 dev test MDW DBG retryq Adding PKT=0x5f2f2a0 to retry queue. retries=1, opaque=19, time=8937us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:49.999493 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:49.999500 dev test MDW DBG ioctx (CTX=0x5f2e760,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.009590 dev test MDW DBG retryq Flush PKT=0x5f2f1a0 to network. retries=1, opaque=18, IX=0, time=19035us 2020/12/21 09:09:50.009603 dev test MDW DBG retryq Flush PKT=0x5f2f2a0 to network. retries=1, opaque=19, IX=0, time=19033us 2020/12/21 09:09:50.009606 dev test MDW DBG retryq Next tick in 10 ms 2020/12/21 09:09:50.010687 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.021769 dev test MDW DBG retryq Flush PKT=0x5f5daa0 to network. retries=3, opaque=15, IX=2, time=86368us 2020/12/21 09:09:50.021783 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.021790 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0bf70 2020/12/21 09:09:50.021796 dev test MDW INFO connection (SOCK=f7a8f15294fc5e77) Starting. Timeout=100000us 2020/12/21 09:09:50.021805 dev test MDW DBG connection (SOCK=f7a8f15294fc5e77) Created new socket with FD=17 2020/12/21 09:09:50.021829 dev test MDW DBG connection (SOCK=f7a8f15294fc5e77) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.022559 dev test MDW INFO connection (SOCK=f7a8f15294fc5e77) Connected established 2020/12/21 09:09:50.022570 dev test MDW DBG connection (SOCK=f7a8f15294fc5e77) Successfully set TCP_NODELAY 2020/12/21 09:09:50.022573 dev test MDW DBG connection (SOCK=f7a8f15294fc5e77) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.022576 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0bf70,C=(nil); E=0x0 2020/12/21 09:09:50.022581 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5fb9f10 SOCKET=0x5f2ef00 2020/12/21 09:09:50.022583 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.022588 dev test MDW DBG ioctx (CTX=0x5f2eb80,unknown) Pairing with SOCK=f7a8f15294fc5e77 2020/12/21 09:09:50.022614 dev test MDW DBG negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f2e8f0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/f7a8f15294fc5e77"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.023547 dev test MDW DBG negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f2e8f0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.024104 dev test MDW DBG negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f2e8f0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.029283 dev test MDW DBG server (CTX=0x5f0ca20,memcached,SRV=0x5f15800,IX=0) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:50.029941 dev test MDW WARN negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f2e8f0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.029969 dev test MDW ERROR [0/] negotiation (CTX=0x5f2eb80,sasl,SASLREQ=0x5f2e8f0) Error: 0xce, SASL Step failed (ref: "9ae23b02-2d1c-42d9-d1a6-83daef47df13") 2020/12/21 09:09:50.029979 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.029984 dev test MDW DBG retryq Adding PKT=0x5f5db30 to retry queue. retries=4, opaque=15, time=94585us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.029986 dev test MDW DBG retryq Next tick in 5 ms 2020/12/21 09:09:50.029989 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.029997 dev test MDW DBG ioctx (CTX=0x5f2eb80,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.030032 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.030040 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.030044 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:50.031130 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.036212 dev test MDW WARN retryq Failing command (pkt=0x5f5db30, opaque=15, retries=4, time=100809us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.036232 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword3" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [100ms] 2020/12/21 09:09:50.036240 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.036244 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.036252 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.036254 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.036257 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.036262 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.036265 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:50.036272 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword3], on master server [node3:11210], global elapsed time: [100ms] 2020/12/21 09:09:50.036298 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.036309 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.036311 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.036313 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.037034 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword4" ; lcb error: LCB_SUCCESS (0): Success (Not an error), global elapsed time: [730us] 2020/12/21 09:09:50.037048 dev test MDW DBG Success Notifying success 2020/12/21 09:09:50.037052 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:50.037054 dev test MDW DBG Success Notifying success 2020/12/21 09:09:50.037057 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:50.037062 dev test MDW DBG TST Stored "ConnectionAndKeyValueCircuitBreakers_ChangePassword4" CAS: 1608541789766418432 on host node1:8091 2020/12/21 09:09:50.037092 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.037102 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.037105 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.037107 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.037120 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.037125 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f5db70 2020/12/21 09:09:50.037130 dev test MDW INFO connection (SOCK=cec5b400816f674f) Starting. Timeout=100000us 2020/12/21 09:09:50.037140 dev test MDW DBG connection (SOCK=cec5b400816f674f) Created new socket with FD=17 2020/12/21 09:09:50.037164 dev test MDW DBG connection (SOCK=cec5b400816f674f) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.037843 dev test MDW INFO connection (SOCK=cec5b400816f674f) Connected established 2020/12/21 09:09:50.037854 dev test MDW DBG connection (SOCK=cec5b400816f674f) Successfully set TCP_NODELAY 2020/12/21 09:09:50.037857 dev test MDW DBG connection (SOCK=cec5b400816f674f) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.037860 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f5db70,C=(nil); E=0x0 2020/12/21 09:09:50.037863 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f2e840 SOCKET=0x5f2ecf0 2020/12/21 09:09:50.037866 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.037871 dev test MDW DBG ioctx (CTX=0x5f2e460,unknown) Pairing with SOCK=cec5b400816f674f 2020/12/21 09:09:50.037897 dev test MDW DBG negotiation (CTX=0x5f2e460,sasl,SASLREQ=0x5f2ea90) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/cec5b400816f674f"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.038883 dev test MDW DBG negotiation (CTX=0x5f2e460,sasl,SASLREQ=0x5f2ea90) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.039405 dev test MDW DBG negotiation (CTX=0x5f2e460,sasl,SASLREQ=0x5f2ea90) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.044577 dev test MDW DBG server (CTX=(nil),,SRV=0x5f0def0,IX=1) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:50.045253 dev test MDW WARN negotiation (CTX=0x5f2e460,sasl,SASLREQ=0x5f2ea90) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.045281 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e460,sasl,SASLREQ=0x5f2ea90) Error: 0xce, SASL Step failed (ref: "d691fedb-8b87-4c07-41d5-5200e3841b8c") 2020/12/21 09:09:50.045291 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.045302 dev test MDW DBG retryq Adding PKT=0x5f2f000 to retry queue. retries=1, opaque=22, time=8188us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.045304 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.045308 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.045315 dev test MDW DBG ioctx (CTX=0x5f2e460,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.045350 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.045358 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.045361 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:50.045372 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.045377 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f0c060 2020/12/21 09:09:50.045382 dev test MDW INFO connection (SOCK=93882d3a455d6805) Starting. Timeout=100000us 2020/12/21 09:09:50.045392 dev test MDW DBG connection (SOCK=93882d3a455d6805) Created new socket with FD=17 2020/12/21 09:09:50.045413 dev test MDW DBG connection (SOCK=93882d3a455d6805) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.045996 dev test MDW INFO connection (SOCK=93882d3a455d6805) Connected established 2020/12/21 09:09:50.046007 dev test MDW DBG connection (SOCK=93882d3a455d6805) Successfully set TCP_NODELAY 2020/12/21 09:09:50.046010 dev test MDW DBG connection (SOCK=93882d3a455d6805) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.046013 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f0c060,C=(nil); E=0x0 2020/12/21 09:09:50.046017 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f0bf70 SOCKET=0x5f2f2e0 2020/12/21 09:09:50.046019 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.046024 dev test MDW DBG ioctx (CTX=0x5f2ec80,unknown) Pairing with SOCK=93882d3a455d6805 2020/12/21 09:09:50.046050 dev test MDW DBG negotiation (CTX=0x5f2ec80,sasl,SASLREQ=0x5f2ea90) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/93882d3a455d6805"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.046847 dev test MDW DBG negotiation (CTX=0x5f2ec80,sasl,SASLREQ=0x5f2ea90) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.047364 dev test MDW DBG negotiation (CTX=0x5f2ec80,sasl,SASLREQ=0x5f2ea90) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.052683 dev test MDW WARN negotiation (CTX=0x5f2ec80,sasl,SASLREQ=0x5f2ea90) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.052711 dev test MDW ERROR [0/] negotiation (CTX=0x5f2ec80,sasl,SASLREQ=0x5f2ea90) Error: 0xce, SASL Step failed (ref: "084931ce-db56-4f0a-cb4f-f88314a5a9bd") 2020/12/21 09:09:50.052721 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.052726 dev test MDW DBG retryq Adding PKT=0x5f0bf70 to retry queue. retries=1, opaque=23, time=7359us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.052729 dev test MDW DBG retryq Next tick in 2 ms 2020/12/21 09:09:50.052735 dev test MDW DBG retryq Adding PKT=0x5fb9a70 to retry queue. retries=1, opaque=24, time=7365us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.052737 dev test MDW DBG retryq Next tick in 2 ms 2020/12/21 09:09:50.052744 dev test MDW DBG ioctx (CTX=0x5f2ec80,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.055832 dev test MDW DBG retryq Flush PKT=0x5f2f000 to network. retries=1, opaque=22, IX=2, time=18717us 2020/12/21 09:09:50.055846 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.055853 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c090 2020/12/21 09:09:50.055858 dev test MDW INFO connection (SOCK=21f2352892fcb389) Starting. Timeout=100000us 2020/12/21 09:09:50.055867 dev test MDW DBG connection (SOCK=21f2352892fcb389) Created new socket with FD=17 2020/12/21 09:09:50.055891 dev test MDW DBG connection (SOCK=21f2352892fcb389) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.055902 dev test MDW DBG retryq Next tick in 6 ms 2020/12/21 09:09:50.056462 dev test MDW INFO connection (SOCK=21f2352892fcb389) Connected established 2020/12/21 09:09:50.056472 dev test MDW DBG connection (SOCK=21f2352892fcb389) Successfully set TCP_NODELAY 2020/12/21 09:09:50.056475 dev test MDW DBG connection (SOCK=21f2352892fcb389) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.056478 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c090,C=(nil); E=0x0 2020/12/21 09:09:50.056482 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f5db70 SOCKET=0x5f2f1f0 2020/12/21 09:09:50.056484 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.056488 dev test MDW DBG ioctx (CTX=0x5f2ed10,unknown) Pairing with SOCK=21f2352892fcb389 2020/12/21 09:09:50.056514 dev test MDW DBG negotiation (CTX=0x5f2ed10,sasl,SASLREQ=0x5f2ea90) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/21f2352892fcb389"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.057389 dev test MDW DBG negotiation (CTX=0x5f2ed10,sasl,SASLREQ=0x5f2ea90) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.057942 dev test MDW DBG negotiation (CTX=0x5f2ed10,sasl,SASLREQ=0x5f2ea90) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.062877 dev test MDW DBG retryq Flush PKT=0x5f0bf70 to network. retries=1, opaque=23, IX=0, time=17506us 2020/12/21 09:09:50.062889 dev test MDW DBG retryq Flush PKT=0x5fb9a70 to network. retries=1, opaque=24, IX=0, time=17505us 2020/12/21 09:09:50.063403 dev test MDW WARN negotiation (CTX=0x5f2ed10,sasl,SASLREQ=0x5f2ea90) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.063430 dev test MDW ERROR [0/] negotiation (CTX=0x5f2ed10,sasl,SASLREQ=0x5f2ea90) Error: 0xce, SASL Step failed (ref: "f435f2b5-9110-4007-f06b-e67570177acc") 2020/12/21 09:09:50.063440 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.063450 dev test MDW DBG retryq Adding PKT=0x5f66080 to retry queue. retries=2, opaque=22, time=26338us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.063452 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:50.063461 dev test MDW DBG ioctx (CTX=0x5f2ed10,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.064043 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.084123 dev test MDW DBG retryq Flush PKT=0x5f66080 to network. retries=2, opaque=22, IX=2, time=47008us 2020/12/21 09:09:50.084138 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.084143 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5fb8af0 2020/12/21 09:09:50.084148 dev test MDW INFO connection (SOCK=27feee23085f5e2e) Starting. Timeout=100000us 2020/12/21 09:09:50.084158 dev test MDW DBG connection (SOCK=27feee23085f5e2e) Created new socket with FD=17 2020/12/21 09:09:50.084181 dev test MDW DBG connection (SOCK=27feee23085f5e2e) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.084798 dev test MDW INFO connection (SOCK=27feee23085f5e2e) Connected established 2020/12/21 09:09:50.084809 dev test MDW DBG connection (SOCK=27feee23085f5e2e) Successfully set TCP_NODELAY 2020/12/21 09:09:50.084820 dev test MDW DBG connection (SOCK=27feee23085f5e2e) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.084823 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5fb8af0,C=(nil); E=0x0 2020/12/21 09:09:50.084826 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f5daa0 SOCKET=0x5fb9b00 2020/12/21 09:09:50.084828 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.084833 dev test MDW DBG ioctx (CTX=0x5f2f160,unknown) Pairing with SOCK=27feee23085f5e2e 2020/12/21 09:09:50.084859 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5f2eef0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/27feee23085f5e2e"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.085693 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5f2eef0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.086256 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5f2eef0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.091689 dev test MDW WARN negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5f2eef0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.091720 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5f2eef0) Error: 0xce, SASL Step failed (ref: "05223bf0-37a2-4022-79ac-d6166dbd37dc") 2020/12/21 09:09:50.091724 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.091730 dev test MDW DBG retryq Adding PKT=0x5fb9a70 to retry queue. retries=3, opaque=22, time=54617us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.091732 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:50.091736 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.091743 dev test MDW DBG ioctx (CTX=0x5f2f160,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.091776 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.091786 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.091789 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:50.091797 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.091802 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c090 2020/12/21 09:09:50.091808 dev test MDW INFO connection (SOCK=02add21a36329962) Starting. Timeout=100000us 2020/12/21 09:09:50.091817 dev test MDW DBG connection (SOCK=02add21a36329962) Created new socket with FD=17 2020/12/21 09:09:50.091837 dev test MDW DBG connection (SOCK=02add21a36329962) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.092532 dev test MDW INFO connection (SOCK=02add21a36329962) Connected established 2020/12/21 09:09:50.092542 dev test MDW DBG connection (SOCK=02add21a36329962) Successfully set TCP_NODELAY 2020/12/21 09:09:50.092545 dev test MDW DBG connection (SOCK=02add21a36329962) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.092548 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c090,C=(nil); E=0x0 2020/12/21 09:09:50.092552 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5fb8af0 SOCKET=0x5f2f080 2020/12/21 09:09:50.092554 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.092559 dev test MDW DBG ioctx (CTX=0x5f0bf70,unknown) Pairing with SOCK=02add21a36329962 2020/12/21 09:09:50.092584 dev test MDW DBG negotiation (CTX=0x5f0bf70,sasl,SASLREQ=0x5f2eea0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/02add21a36329962"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.093487 dev test MDW DBG negotiation (CTX=0x5f0bf70,sasl,SASLREQ=0x5f2eea0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.094039 dev test MDW DBG negotiation (CTX=0x5f0bf70,sasl,SASLREQ=0x5f2eea0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.099707 dev test MDW WARN negotiation (CTX=0x5f0bf70,sasl,SASLREQ=0x5f2eea0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.099734 dev test MDW ERROR [0/] negotiation (CTX=0x5f0bf70,sasl,SASLREQ=0x5f2eea0) Error: 0xce, SASL Step failed (ref: "37a3194f-5119-4ee7-45cd-aafdc4a6763d") 2020/12/21 09:09:50.099739 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.099744 dev test MDW DBG retryq Adding PKT=0x5fb9b00 to retry queue. retries=1, opaque=25, time=7950us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.099746 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.099750 dev test MDW DBG retryq Adding PKT=0x5f2e780 to retry queue. retries=1, opaque=26, time=7955us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.099752 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.099760 dev test MDW DBG ioctx (CTX=0x5f0bf70,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.109847 dev test MDW DBG retryq Flush PKT=0x5fb9b00 to network. retries=1, opaque=25, IX=0, time=18051us 2020/12/21 09:09:50.109860 dev test MDW DBG retryq Flush PKT=0x5f2e780 to network. retries=1, opaque=26, IX=0, time=18050us 2020/12/21 09:09:50.109862 dev test MDW DBG retryq Next tick in 11 ms 2020/12/21 09:09:50.111080 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.123161 dev test MDW DBG retryq Flush PKT=0x5fb9a70 to network. retries=3, opaque=22, IX=2, time=86045us 2020/12/21 09:09:50.123178 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.123184 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0bf70 2020/12/21 09:09:50.123209 dev test MDW INFO connection (SOCK=171a3cb3eea03a95) Starting. Timeout=100000us 2020/12/21 09:09:50.123221 dev test MDW DBG connection (SOCK=171a3cb3eea03a95) Created new socket with FD=17 2020/12/21 09:09:50.123248 dev test MDW DBG connection (SOCK=171a3cb3eea03a95) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.124000 dev test MDW INFO connection (SOCK=171a3cb3eea03a95) Connected established 2020/12/21 09:09:50.124011 dev test MDW DBG connection (SOCK=171a3cb3eea03a95) Successfully set TCP_NODELAY 2020/12/21 09:09:50.124015 dev test MDW DBG connection (SOCK=171a3cb3eea03a95) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.124017 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0bf70,C=(nil); E=0x0 2020/12/21 09:09:50.124020 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f5db70 SOCKET=0x5f2e810 2020/12/21 09:09:50.124023 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.124028 dev test MDW DBG ioctx (CTX=0x5f2f160,unknown) Pairing with SOCK=171a3cb3eea03a95 2020/12/21 09:09:50.124054 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/171a3cb3eea03a95"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.125009 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.125538 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.131345 dev test MDW WARN negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.131373 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) Error: 0xce, SASL Step failed (ref: "9f235338-6aaf-4c03-d189-89f0c0653259") 2020/12/21 09:09:50.131383 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.131394 dev test MDW DBG retryq Adding PKT=0x5f2e880 to retry queue. retries=4, opaque=22, time=94281us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.131395 dev test MDW DBG retryq Next tick in 5 ms 2020/12/21 09:09:50.131399 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.131406 dev test MDW DBG ioctx (CTX=0x5f2f160,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.131441 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.131449 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.131452 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:50.132482 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.137562 dev test MDW WARN retryq Failing command (pkt=0x5f2e880, opaque=22, retries=4, time=100447us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.137582 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword5" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [100ms] 2020/12/21 09:09:50.137590 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.137595 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.137602 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.137606 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.137608 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.137613 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.137616 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:50.137623 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword5], on master server [node3:11210], global elapsed time: [100ms] 2020/12/21 09:09:50.137651 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.137661 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.137663 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.137666 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.137680 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.137685 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f5daa0 2020/12/21 09:09:50.137690 dev test MDW INFO connection (SOCK=118358eb149030f7) Starting. Timeout=100000us 2020/12/21 09:09:50.137699 dev test MDW DBG connection (SOCK=118358eb149030f7) Created new socket with FD=17 2020/12/21 09:09:50.137723 dev test MDW DBG connection (SOCK=118358eb149030f7) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.138410 dev test MDW INFO connection (SOCK=118358eb149030f7) Connected established 2020/12/21 09:09:50.138420 dev test MDW DBG connection (SOCK=118358eb149030f7) Successfully set TCP_NODELAY 2020/12/21 09:09:50.138424 dev test MDW DBG connection (SOCK=118358eb149030f7) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.138426 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f5daa0,C=(nil); E=0x0 2020/12/21 09:09:50.138431 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5fb8a90 SOCKET=0x5f2f0f0 2020/12/21 09:09:50.138433 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.138438 dev test MDW DBG ioctx (CTX=0x5f2f160,unknown) Pairing with SOCK=118358eb149030f7 2020/12/21 09:09:50.138465 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/118358eb149030f7"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.139400 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.139934 dev test MDW DBG negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.145621 dev test MDW WARN negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.145649 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f160,sasl,SASLREQ=0x5fb9d30) Error: 0xce, SASL Step failed (ref: "a86dcf77-6783-4b22-b594-1e14517081b7") 2020/12/21 09:09:50.145655 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.145660 dev test MDW DBG retryq Adding PKT=0x5fb9ec0 to retry queue. retries=1, opaque=28, time=7989us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.145662 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.145665 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.145673 dev test MDW DBG ioctx (CTX=0x5f2f160,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.145708 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.145717 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.145720 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:50.145730 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.145736 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f5db30 2020/12/21 09:09:50.145741 dev test MDW INFO connection (SOCK=15a8589065ef16c5) Starting. Timeout=100000us 2020/12/21 09:09:50.145751 dev test MDW DBG connection (SOCK=15a8589065ef16c5) Created new socket with FD=17 2020/12/21 09:09:50.145772 dev test MDW DBG connection (SOCK=15a8589065ef16c5) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.146346 dev test MDW INFO connection (SOCK=15a8589065ef16c5) Connected established 2020/12/21 09:09:50.146356 dev test MDW DBG connection (SOCK=15a8589065ef16c5) Successfully set TCP_NODELAY 2020/12/21 09:09:50.146360 dev test MDW DBG connection (SOCK=15a8589065ef16c5) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.146362 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f5db30,C=(nil); E=0x0 2020/12/21 09:09:50.146366 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5dce0 SOCKET=0x5fb9dc0 2020/12/21 09:09:50.146369 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.146373 dev test MDW DBG ioctx (CTX=0x5f2e550,unknown) Pairing with SOCK=15a8589065ef16c5 2020/12/21 09:09:50.146399 dev test MDW DBG negotiation (CTX=0x5f2e550,sasl,SASLREQ=0x5f2e9f0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/15a8589065ef16c5"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.147339 dev test MDW DBG negotiation (CTX=0x5f2e550,sasl,SASLREQ=0x5f2e9f0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.147846 dev test MDW DBG negotiation (CTX=0x5f2e550,sasl,SASLREQ=0x5f2e9f0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.153335 dev test MDW WARN negotiation (CTX=0x5f2e550,sasl,SASLREQ=0x5f2e9f0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.153363 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e550,sasl,SASLREQ=0x5f2e9f0) Error: 0xce, SASL Step failed (ref: "745d0144-c9ff-48e3-5fc3-61ec744c251c") 2020/12/21 09:09:50.153367 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.153372 dev test MDW DBG retryq Adding PKT=0x5fb9d30 to retry queue. retries=1, opaque=29, time=7646us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.153374 dev test MDW DBG retryq Next tick in 2 ms 2020/12/21 09:09:50.153378 dev test MDW DBG retryq Adding PKT=0x5fb9e30 to retry queue. retries=1, opaque=30, time=7651us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.153380 dev test MDW DBG retryq Next tick in 2 ms 2020/12/21 09:09:50.153387 dev test MDW DBG ioctx (CTX=0x5f2e550,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.156483 dev test MDW DBG retryq Flush PKT=0x5fb9ec0 to network. retries=1, opaque=28, IX=1, time=18809us 2020/12/21 09:09:50.156498 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.156503 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f2e880 2020/12/21 09:09:50.156508 dev test MDW INFO connection (SOCK=e6d34395c8a41d29) Starting. Timeout=100000us 2020/12/21 09:09:50.156517 dev test MDW DBG connection (SOCK=e6d34395c8a41d29) Created new socket with FD=17 2020/12/21 09:09:50.156538 dev test MDW DBG connection (SOCK=e6d34395c8a41d29) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.156543 dev test MDW DBG retryq Next tick in 6 ms 2020/12/21 09:09:50.157271 dev test MDW INFO connection (SOCK=e6d34395c8a41d29) Connected established 2020/12/21 09:09:50.157281 dev test MDW DBG connection (SOCK=e6d34395c8a41d29) Successfully set TCP_NODELAY 2020/12/21 09:09:50.157285 dev test MDW DBG connection (SOCK=e6d34395c8a41d29) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.157287 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f2e880,C=(nil); E=0x0 2020/12/21 09:09:50.157290 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5dce0 SOCKET=0x5f2e9f0 2020/12/21 09:09:50.157293 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.157298 dev test MDW DBG ioctx (CTX=0x5f2e5c0,unknown) Pairing with SOCK=e6d34395c8a41d29 2020/12/21 09:09:50.157323 dev test MDW DBG negotiation (CTX=0x5f2e5c0,sasl,SASLREQ=0x5f2e460) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/e6d34395c8a41d29"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.158381 dev test MDW DBG negotiation (CTX=0x5f2e5c0,sasl,SASLREQ=0x5f2e460) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.158936 dev test MDW DBG negotiation (CTX=0x5f2e5c0,sasl,SASLREQ=0x5f2e460) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.164086 dev test MDW DBG retryq Flush PKT=0x5fb9d30 to network. retries=1, opaque=29, IX=0, time=18357us 2020/12/21 09:09:50.164099 dev test MDW DBG retryq Flush PKT=0x5fb9e30 to network. retries=1, opaque=30, IX=0, time=18355us 2020/12/21 09:09:50.165178 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.165202 dev test MDW WARN negotiation (CTX=0x5f2e5c0,sasl,SASLREQ=0x5f2e460) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.165224 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e5c0,sasl,SASLREQ=0x5f2e460) Error: 0xce, SASL Step failed (ref: "3f80eea5-7aa1-4c33-0eaa-5b837a33e9d2") 2020/12/21 09:09:50.165234 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.165242 dev test MDW DBG retryq Adding PKT=0x5fb9e30 to retry queue. retries=2, opaque=28, time=27570us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.165244 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:50.165247 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.165254 dev test MDW DBG ioctx (CTX=0x5f2e5c0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.165289 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.165297 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.165300 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:50.165309 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.165315 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f0c000 2020/12/21 09:09:50.165320 dev test MDW INFO connection (SOCK=bc6ca27dca9dea8e) Starting. Timeout=100000us 2020/12/21 09:09:50.165329 dev test MDW DBG connection (SOCK=bc6ca27dca9dea8e) Created new socket with FD=17 2020/12/21 09:09:50.165357 dev test MDW DBG connection (SOCK=bc6ca27dca9dea8e) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.166048 dev test MDW INFO connection (SOCK=bc6ca27dca9dea8e) Connected established 2020/12/21 09:09:50.166058 dev test MDW DBG connection (SOCK=bc6ca27dca9dea8e) Successfully set TCP_NODELAY 2020/12/21 09:09:50.166061 dev test MDW DBG connection (SOCK=bc6ca27dca9dea8e) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.166064 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f0c000,C=(nil); E=0x0 2020/12/21 09:09:50.166067 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f5dce0 SOCKET=0x5f5db30 2020/12/21 09:09:50.166069 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.166074 dev test MDW DBG ioctx (CTX=0x5f2e4c0,unknown) Pairing with SOCK=bc6ca27dca9dea8e 2020/12/21 09:09:50.166100 dev test MDW DBG negotiation (CTX=0x5f2e4c0,sasl,SASLREQ=0x5f2e810) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/bc6ca27dca9dea8e"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.167231 dev test MDW DBG negotiation (CTX=0x5f2e4c0,sasl,SASLREQ=0x5f2e810) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.167780 dev test MDW DBG negotiation (CTX=0x5f2e4c0,sasl,SASLREQ=0x5f2e810) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.173723 dev test MDW WARN negotiation (CTX=0x5f2e4c0,sasl,SASLREQ=0x5f2e810) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.173751 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e4c0,sasl,SASLREQ=0x5f2e810) Error: 0xce, SASL Step failed (ref: "0ad92271-248b-4749-934c-f723bf208795") 2020/12/21 09:09:50.173755 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.173760 dev test MDW DBG retryq Adding PKT=0x5fb8350 to retry queue. retries=1, opaque=31, time=8454us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.173762 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.173766 dev test MDW DBG retryq Adding PKT=0x5fb83e0 to retry queue. retries=1, opaque=32, time=8460us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.173768 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.173775 dev test MDW DBG ioctx (CTX=0x5f2e4c0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.183867 dev test MDW DBG retryq Flush PKT=0x5fb8350 to network. retries=1, opaque=31, IX=0, time=18559us 2020/12/21 09:09:50.183881 dev test MDW DBG retryq Flush PKT=0x5fb83e0 to network. retries=1, opaque=32, IX=0, time=18557us 2020/12/21 09:09:50.183883 dev test MDW DBG retryq Flush PKT=0x5fb9e30 to network. retries=2, opaque=28, IX=1, time=46193us 2020/12/21 09:09:50.183889 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.183894 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f2e460 2020/12/21 09:09:50.183907 dev test MDW INFO connection (SOCK=cf45c3fa5e9e9862) Starting. Timeout=100000us 2020/12/21 09:09:50.183917 dev test MDW DBG connection (SOCK=cf45c3fa5e9e9862) Created new socket with FD=17 2020/12/21 09:09:50.183940 dev test MDW DBG connection (SOCK=cf45c3fa5e9e9862) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.184589 dev test MDW INFO connection (SOCK=cf45c3fa5e9e9862) Connected established 2020/12/21 09:09:50.184601 dev test MDW DBG connection (SOCK=cf45c3fa5e9e9862) Successfully set TCP_NODELAY 2020/12/21 09:09:50.184604 dev test MDW DBG connection (SOCK=cf45c3fa5e9e9862) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.184607 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f2e460,C=(nil); E=0x0 2020/12/21 09:09:50.184612 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f0c000 SOCKET=0x5f2e760 2020/12/21 09:09:50.184614 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.184619 dev test MDW DBG ioctx (CTX=0x5f5daa0,unknown) Pairing with SOCK=cf45c3fa5e9e9862 2020/12/21 09:09:50.184645 dev test MDW DBG negotiation (CTX=0x5f5daa0,sasl,SASLREQ=0x5f2e810) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/cf45c3fa5e9e9862"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.185039 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.185860 dev test MDW DBG negotiation (CTX=0x5f5daa0,sasl,SASLREQ=0x5f2e810) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.186409 dev test MDW DBG negotiation (CTX=0x5f5daa0,sasl,SASLREQ=0x5f2e810) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.191950 dev test MDW WARN negotiation (CTX=0x5f5daa0,sasl,SASLREQ=0x5f2e810) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.191978 dev test MDW ERROR [0/] negotiation (CTX=0x5f5daa0,sasl,SASLREQ=0x5f2e810) Error: 0xce, SASL Step failed (ref: "a7b0ea5d-0094-42fe-36bd-2d1c2d71d33c") 2020/12/21 09:09:50.191983 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.191989 dev test MDW DBG retryq Adding PKT=0x5fb7ac0 to retry queue. retries=3, opaque=28, time=54317us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.191991 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:50.191994 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.192002 dev test MDW DBG ioctx (CTX=0x5f5daa0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.192038 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.192047 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.192050 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:50.192986 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.223070 dev test MDW DBG retryq Flush PKT=0x5fb7ac0 to network. retries=3, opaque=28, IX=1, time=85396us 2020/12/21 09:09:50.223086 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.223092 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f0bf70 2020/12/21 09:09:50.223097 dev test MDW INFO connection (SOCK=1971a0f2fa5aeb1d) Starting. Timeout=100000us 2020/12/21 09:09:50.223107 dev test MDW DBG connection (SOCK=1971a0f2fa5aeb1d) Created new socket with FD=17 2020/12/21 09:09:50.223130 dev test MDW DBG connection (SOCK=1971a0f2fa5aeb1d) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.223831 dev test MDW INFO connection (SOCK=1971a0f2fa5aeb1d) Connected established 2020/12/21 09:09:50.223842 dev test MDW DBG connection (SOCK=1971a0f2fa5aeb1d) Successfully set TCP_NODELAY 2020/12/21 09:09:50.223846 dev test MDW DBG connection (SOCK=1971a0f2fa5aeb1d) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.223849 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f0bf70,C=(nil); E=0x0 2020/12/21 09:09:50.223852 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f5dce0 SOCKET=0x5f2e8a0 2020/12/21 09:09:50.223854 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.223859 dev test MDW DBG ioctx (CTX=0x5f2e520,unknown) Pairing with SOCK=1971a0f2fa5aeb1d 2020/12/21 09:09:50.223885 dev test MDW DBG negotiation (CTX=0x5f2e520,sasl,SASLREQ=0x5fb9e50) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/1971a0f2fa5aeb1d"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.224789 dev test MDW DBG negotiation (CTX=0x5f2e520,sasl,SASLREQ=0x5fb9e50) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.225346 dev test MDW DBG negotiation (CTX=0x5f2e520,sasl,SASLREQ=0x5fb9e50) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.230993 dev test MDW WARN negotiation (CTX=0x5f2e520,sasl,SASLREQ=0x5fb9e50) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.231021 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e520,sasl,SASLREQ=0x5fb9e50) Error: 0xce, SASL Step failed (ref: "5cbe1baa-fa6d-4b35-e5e7-e44b963c9ec9") 2020/12/21 09:09:50.231026 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.231031 dev test MDW DBG retryq Adding PKT=0x5f5daa0 to retry queue. retries=4, opaque=28, time=93360us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.231034 dev test MDW DBG retryq Next tick in 6 ms 2020/12/21 09:09:50.231038 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.231047 dev test MDW DBG ioctx (CTX=0x5f2e520,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.231082 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.231091 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.231094 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:50.231104 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.231111 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f0bfb0 2020/12/21 09:09:50.231117 dev test MDW INFO connection (SOCK=d23faf55424a4c8c) Starting. Timeout=100000us 2020/12/21 09:09:50.231128 dev test MDW DBG connection (SOCK=d23faf55424a4c8c) Created new socket with FD=17 2020/12/21 09:09:50.231149 dev test MDW DBG connection (SOCK=d23faf55424a4c8c) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.231892 dev test MDW INFO connection (SOCK=d23faf55424a4c8c) Connected established 2020/12/21 09:09:50.231903 dev test MDW DBG connection (SOCK=d23faf55424a4c8c) Successfully set TCP_NODELAY 2020/12/21 09:09:50.231907 dev test MDW DBG connection (SOCK=d23faf55424a4c8c) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.231909 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f0bfb0,C=(nil); E=0x0 2020/12/21 09:09:50.231912 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5fb9d30 SOCKET=0x5fb9f40 2020/12/21 09:09:50.231915 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.231920 dev test MDW DBG ioctx (CTX=0x5f2e770,unknown) Pairing with SOCK=d23faf55424a4c8c 2020/12/21 09:09:50.231946 dev test MDW DBG negotiation (CTX=0x5f2e770,sasl,SASLREQ=0x5f2e4f0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/d23faf55424a4c8c"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.232935 dev test MDW DBG negotiation (CTX=0x5f2e770,sasl,SASLREQ=0x5f2e4f0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.233412 dev test MDW DBG negotiation (CTX=0x5f2e770,sasl,SASLREQ=0x5f2e4f0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.238494 dev test MDW WARN retryq Failing command (pkt=0x5f5daa0, opaque=28, retries=4, time=100818us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.238516 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword6" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [100ms] 2020/12/21 09:09:50.238524 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.238529 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.238537 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.238540 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.238546 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.238551 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.238554 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:50.238562 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword6], on master server [node2:11210], global elapsed time: [100ms] 2020/12/21 09:09:50.239121 dev test MDW WARN negotiation (CTX=0x5f2e770,sasl,SASLREQ=0x5f2e4f0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.239149 dev test MDW ERROR [0/] negotiation (CTX=0x5f2e770,sasl,SASLREQ=0x5f2e4f0) Error: 0xce, SASL Step failed (ref: "18002827-efcc-4a91-2651-d8c3eb00c135") 2020/12/21 09:09:50.239159 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.239165 dev test MDW DBG retryq Adding PKT=0x5f5daa0 to retry queue. retries=1, opaque=34, time=8066us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.239167 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.239171 dev test MDW DBG retryq Adding PKT=0x5f5db30 to retry queue. retries=1, opaque=35, time=8071us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.239173 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.239181 dev test MDW DBG ioctx (CTX=0x5f2e770,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.249271 dev test MDW DBG retryq Flush PKT=0x5f5daa0 to network. retries=1, opaque=34, IX=0, time=18168us 2020/12/21 09:09:50.249285 dev test MDW DBG retryq Flush PKT=0x5f5db30 to network. retries=1, opaque=35, IX=0, time=18167us 2020/12/21 09:09:50.250302 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.250349 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.250359 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.250362 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.250364 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.250378 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.250384 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f226f0 2020/12/21 09:09:50.250390 dev test MDW INFO connection (SOCK=c24ebc6cbc606e7f) Starting. Timeout=100000us 2020/12/21 09:09:50.250401 dev test MDW DBG connection (SOCK=c24ebc6cbc606e7f) Created new socket with FD=17 2020/12/21 09:09:50.250424 dev test MDW DBG connection (SOCK=c24ebc6cbc606e7f) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.251050 dev test MDW INFO connection (SOCK=c24ebc6cbc606e7f) Connected established 2020/12/21 09:09:50.251061 dev test MDW DBG connection (SOCK=c24ebc6cbc606e7f) Successfully set TCP_NODELAY 2020/12/21 09:09:50.251065 dev test MDW DBG connection (SOCK=c24ebc6cbc606e7f) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.251067 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f226f0,C=(nil); E=0x0 2020/12/21 09:09:50.251071 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f0bfb0 SOCKET=0x5f2f3a0 2020/12/21 09:09:50.251073 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.251078 dev test MDW DBG ioctx (CTX=0x5f23590,unknown) Pairing with SOCK=c24ebc6cbc606e7f 2020/12/21 09:09:50.251104 dev test MDW DBG negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f23310) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/c24ebc6cbc606e7f"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.252000 dev test MDW DBG negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f23310) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.252573 dev test MDW DBG negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f23310) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.258161 dev test MDW WARN negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f23310) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.258190 dev test MDW ERROR [0/] negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f23310) Error: 0xce, SASL Step failed (ref: "61875f87-28ab-4ebf-b167-f81a8e83c70b") 2020/12/21 09:09:50.258194 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.258199 dev test MDW DBG retryq Adding PKT=0x5f23c90 to retry queue. retries=1, opaque=36, time=7830us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.258201 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.258204 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.258212 dev test MDW DBG ioctx (CTX=0x5f23590,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.258246 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.258255 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.258258 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:50.258273 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.258278 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f23530 2020/12/21 09:09:50.258284 dev test MDW INFO connection (SOCK=e48a05bdcaabc7c3) Starting. Timeout=100000us 2020/12/21 09:09:50.258294 dev test MDW DBG connection (SOCK=e48a05bdcaabc7c3) Created new socket with FD=17 2020/12/21 09:09:50.258321 dev test MDW DBG connection (SOCK=e48a05bdcaabc7c3) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.259003 dev test MDW INFO connection (SOCK=e48a05bdcaabc7c3) Connected established 2020/12/21 09:09:50.259013 dev test MDW DBG connection (SOCK=e48a05bdcaabc7c3) Successfully set TCP_NODELAY 2020/12/21 09:09:50.259017 dev test MDW DBG connection (SOCK=e48a05bdcaabc7c3) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.259019 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f23530,C=(nil); E=0x0 2020/12/21 09:09:50.259022 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f226f0 SOCKET=0x5f23830 2020/12/21 09:09:50.259024 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.259029 dev test MDW DBG ioctx (CTX=0x5f23590,unknown) Pairing with SOCK=e48a05bdcaabc7c3 2020/12/21 09:09:50.259056 dev test MDW DBG negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f5db30) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/e48a05bdcaabc7c3"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.260002 dev test MDW DBG negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f5db30) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.260517 dev test MDW DBG negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f5db30) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.266335 dev test MDW WARN negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f5db30) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.266365 dev test MDW ERROR [0/] negotiation (CTX=0x5f23590,sasl,SASLREQ=0x5f5db30) Error: 0xce, SASL Step failed (ref: "7617ce08-3c35-4289-77e1-772370aaed61") 2020/12/21 09:09:50.266370 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.266375 dev test MDW DBG retryq Adding PKT=0x5f25af0 to retry queue. retries=1, opaque=37, time=8106us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.266377 dev test MDW DBG retryq Next tick in 1 ms 2020/12/21 09:09:50.266381 dev test MDW DBG retryq Adding PKT=0x5f233e0 to retry queue. retries=1, opaque=38, time=8112us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.266383 dev test MDW DBG retryq Next tick in 1 ms 2020/12/21 09:09:50.266392 dev test MDW DBG ioctx (CTX=0x5f23590,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.268487 dev test MDW DBG retryq Flush PKT=0x5f23c90 to network. retries=1, opaque=36, IX=1, time=18115us 2020/12/21 09:09:50.268502 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.268508 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f2f3a0 2020/12/21 09:09:50.268514 dev test MDW INFO connection (SOCK=a4760c04b102c891) Starting. Timeout=100000us 2020/12/21 09:09:50.268523 dev test MDW DBG connection (SOCK=a4760c04b102c891) Created new socket with FD=17 2020/12/21 09:09:50.268544 dev test MDW DBG connection (SOCK=a4760c04b102c891) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.268556 dev test MDW DBG retryq Next tick in 7 ms 2020/12/21 09:09:50.269269 dev test MDW INFO connection (SOCK=a4760c04b102c891) Connected established 2020/12/21 09:09:50.269280 dev test MDW DBG connection (SOCK=a4760c04b102c891) Successfully set TCP_NODELAY 2020/12/21 09:09:50.269283 dev test MDW DBG connection (SOCK=a4760c04b102c891) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.269286 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f2f3a0,C=(nil); E=0x0 2020/12/21 09:09:50.269289 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f23520 SOCKET=0x5f231f0 2020/12/21 09:09:50.269291 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.269296 dev test MDW DBG ioctx (CTX=0x5f23a10,unknown) Pairing with SOCK=a4760c04b102c891 2020/12/21 09:09:50.269322 dev test MDW DBG negotiation (CTX=0x5f23a10,sasl,SASLREQ=0x5f25950) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/a4760c04b102c891"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.270320 dev test MDW DBG negotiation (CTX=0x5f23a10,sasl,SASLREQ=0x5f25950) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.270865 dev test MDW DBG negotiation (CTX=0x5f23a10,sasl,SASLREQ=0x5f25950) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.276731 dev test MDW WARN negotiation (CTX=0x5f23a10,sasl,SASLREQ=0x5f25950) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.276759 dev test MDW ERROR [0/] negotiation (CTX=0x5f23a10,sasl,SASLREQ=0x5f25950) Error: 0xce, SASL Step failed (ref: "588ce692-c5d1-40c7-1095-86eb0f53c9a7") 2020/12/21 09:09:50.276763 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.276769 dev test MDW DBG retryq Adding PKT=0x5f226a0 to retry queue. retries=2, opaque=36, time=26400us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.276771 dev test MDW DBG retryq Next tick in 0 ms 2020/12/21 09:09:50.276779 dev test MDW DBG ioctx (CTX=0x5f23a10,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.276804 dev test MDW DBG retryq Flush PKT=0x5f25af0 to network. retries=1, opaque=37, IX=0, time=18535us 2020/12/21 09:09:50.276825 dev test MDW DBG retryq Flush PKT=0x5f233e0 to network. retries=1, opaque=38, IX=0, time=18533us 2020/12/21 09:09:50.276828 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:50.277888 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.292970 dev test MDW DBG server (CTX=0x5f0ca20,memcached,SRV=0x5f15800,IX=0) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:50.297049 dev test MDW DBG retryq Flush PKT=0x5f226a0 to network. retries=2, opaque=36, IX=1, time=46678us 2020/12/21 09:09:50.297061 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.297068 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23140 2020/12/21 09:09:50.297073 dev test MDW INFO connection (SOCK=1d83e5b6252542b0) Starting. Timeout=100000us 2020/12/21 09:09:50.297083 dev test MDW DBG connection (SOCK=1d83e5b6252542b0) Created new socket with FD=17 2020/12/21 09:09:50.297106 dev test MDW DBG connection (SOCK=1d83e5b6252542b0) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.297687 dev test MDW INFO connection (SOCK=1d83e5b6252542b0) Connected established 2020/12/21 09:09:50.297697 dev test MDW DBG connection (SOCK=1d83e5b6252542b0) Successfully set TCP_NODELAY 2020/12/21 09:09:50.297700 dev test MDW DBG connection (SOCK=1d83e5b6252542b0) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.297702 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23140,C=(nil); E=0x0 2020/12/21 09:09:50.297705 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f23050 SOCKET=0x5f23470 2020/12/21 09:09:50.297708 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.297713 dev test MDW DBG ioctx (CTX=0x5f2f040,unknown) Pairing with SOCK=1d83e5b6252542b0 2020/12/21 09:09:50.297739 dev test MDW DBG negotiation (CTX=0x5f2f040,sasl,SASLREQ=0x5f2edc0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/1d83e5b6252542b0"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.298532 dev test MDW DBG negotiation (CTX=0x5f2f040,sasl,SASLREQ=0x5f2edc0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.299085 dev test MDW DBG negotiation (CTX=0x5f2f040,sasl,SASLREQ=0x5f2edc0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.304629 dev test MDW WARN negotiation (CTX=0x5f2f040,sasl,SASLREQ=0x5f2edc0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.304657 dev test MDW ERROR [0/] negotiation (CTX=0x5f2f040,sasl,SASLREQ=0x5f2edc0) Error: 0xce, SASL Step failed (ref: "33dcfafb-7bc7-47a7-f7ee-59fa027388e5") 2020/12/21 09:09:50.304661 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.304667 dev test MDW DBG retryq Adding PKT=0x5f233b0 to retry queue. retries=3, opaque=36, time=54298us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.304671 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:50.304680 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.304687 dev test MDW DBG ioctx (CTX=0x5f2f040,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.304721 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.304730 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.304733 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:50.305914 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.335994 dev test MDW DBG retryq Flush PKT=0x5f233b0 to network. retries=3, opaque=36, IX=1, time=85622us 2020/12/21 09:09:50.336012 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.336017 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23ae0 2020/12/21 09:09:50.336021 dev test MDW INFO connection (SOCK=922757e9faf13d62) Starting. Timeout=100000us 2020/12/21 09:09:50.336032 dev test MDW DBG connection (SOCK=922757e9faf13d62) Created new socket with FD=17 2020/12/21 09:09:50.336058 dev test MDW DBG connection (SOCK=922757e9faf13d62) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.336622 dev test MDW INFO connection (SOCK=922757e9faf13d62) Connected established 2020/12/21 09:09:50.336632 dev test MDW DBG connection (SOCK=922757e9faf13d62) Successfully set TCP_NODELAY 2020/12/21 09:09:50.336636 dev test MDW DBG connection (SOCK=922757e9faf13d62) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.336638 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23ae0,C=(nil); E=0x0 2020/12/21 09:09:50.336641 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f2f3a0 SOCKET=0x5f22660 2020/12/21 09:09:50.336645 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.336650 dev test MDW DBG ioctx (CTX=0x5f23440,unknown) Pairing with SOCK=922757e9faf13d62 2020/12/21 09:09:50.336676 dev test MDW DBG negotiation (CTX=0x5f23440,sasl,SASLREQ=0x5f227a0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/922757e9faf13d62"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.337491 dev test MDW DBG negotiation (CTX=0x5f23440,sasl,SASLREQ=0x5f227a0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.338012 dev test MDW DBG negotiation (CTX=0x5f23440,sasl,SASLREQ=0x5f227a0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.343559 dev test MDW WARN negotiation (CTX=0x5f23440,sasl,SASLREQ=0x5f227a0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.343587 dev test MDW ERROR [0/] negotiation (CTX=0x5f23440,sasl,SASLREQ=0x5f227a0) Error: 0xce, SASL Step failed (ref: "58a80812-e853-4abd-abba-4608fde22132") 2020/12/21 09:09:50.343592 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.343597 dev test MDW DBG retryq Adding PKT=0x5f231e0 to retry queue. retries=4, opaque=36, time=93228us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.343601 dev test MDW DBG retryq Next tick in 6 ms 2020/12/21 09:09:50.343605 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.343617 dev test MDW DBG ioctx (CTX=0x5f23440,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.343653 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.343662 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.343665 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:50.343678 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.343684 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23a00 2020/12/21 09:09:50.343689 dev test MDW INFO connection (SOCK=5a8806db1ef0ac33) Starting. Timeout=100000us 2020/12/21 09:09:50.343700 dev test MDW DBG connection (SOCK=5a8806db1ef0ac33) Created new socket with FD=17 2020/12/21 09:09:50.343722 dev test MDW DBG connection (SOCK=5a8806db1ef0ac33) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.344520 dev test MDW INFO connection (SOCK=5a8806db1ef0ac33) Connected established 2020/12/21 09:09:50.344530 dev test MDW DBG connection (SOCK=5a8806db1ef0ac33) Successfully set TCP_NODELAY 2020/12/21 09:09:50.344534 dev test MDW DBG connection (SOCK=5a8806db1ef0ac33) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.344536 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23a00,C=(nil); E=0x0 2020/12/21 09:09:50.344540 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f23ae0 SOCKET=0x5f236e0 2020/12/21 09:09:50.344543 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.344547 dev test MDW DBG ioctx (CTX=0x5f23750,unknown) Pairing with SOCK=5a8806db1ef0ac33 2020/12/21 09:09:50.344573 dev test MDW DBG negotiation (CTX=0x5f23750,sasl,SASLREQ=0x5f23390) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/5a8806db1ef0ac33"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.345672 dev test MDW DBG negotiation (CTX=0x5f23750,sasl,SASLREQ=0x5f23390) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.346217 dev test MDW DBG negotiation (CTX=0x5f23750,sasl,SASLREQ=0x5f23390) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.351453 dev test MDW WARN retryq Failing command (pkt=0x5f231e0, opaque=36, retries=4, time=101079us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.351475 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword7" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [101ms] 2020/12/21 09:09:50.351482 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.351487 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.351494 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.351498 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.351506 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.351512 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.351515 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:50.351523 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword7], on master server [node2:11210], global elapsed time: [101ms] 2020/12/21 09:09:50.352183 dev test MDW WARN negotiation (CTX=0x5f23750,sasl,SASLREQ=0x5f23390) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.352210 dev test MDW ERROR [0/] negotiation (CTX=0x5f23750,sasl,SASLREQ=0x5f23390) Error: 0xce, SASL Step failed (ref: "880aed1d-95e1-4255-5406-7d09d7d43628") 2020/12/21 09:09:50.352219 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.352224 dev test MDW DBG retryq Adding PKT=0x5f23540 to retry queue. retries=1, opaque=40, time=8550us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.352226 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.352230 dev test MDW DBG retryq Adding PKT=0x5f23be0 to retry queue. retries=1, opaque=41, time=8555us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.352232 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.352239 dev test MDW DBG ioctx (CTX=0x5f23750,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.359327 dev test MDW DBG server (CTX=(nil),,SRV=0x5f016c0,IX=2) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:50.362401 dev test MDW DBG retryq Flush PKT=0x5f23540 to network. retries=1, opaque=40, IX=0, time=18726us 2020/12/21 09:09:50.362412 dev test MDW DBG retryq Flush PKT=0x5f23be0 to network. retries=1, opaque=41, IX=0, time=18724us 2020/12/21 09:09:50.364062 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.364104 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.364114 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.364117 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.364119 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.364767 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword8" ; lcb error: LCB_SUCCESS (0): Success (Not an error), global elapsed time: [658us] 2020/12/21 09:09:50.364781 dev test MDW DBG Success Notifying success 2020/12/21 09:09:50.364786 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:50.364788 dev test MDW DBG Success Notifying success 2020/12/21 09:09:50.364791 dev test MDW DBG onSuccess Trying to move from state Closed to Closed 2020/12/21 09:09:50.364796 dev test MDW DBG TST Stored "ConnectionAndKeyValueCircuitBreakers_ChangePassword8" CAS: 1608541790094163968 on host node1:8091 2020/12/21 09:09:50.364836 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.364847 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.364849 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.364852 dev test MDW DBG onEvaluate Trying to move from state Closed to Closed 2020/12/21 09:09:50.364865 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.364871 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23770 2020/12/21 09:09:50.364876 dev test MDW INFO connection (SOCK=cef3da3af8b0de9b) Starting. Timeout=100000us 2020/12/21 09:09:50.364886 dev test MDW DBG connection (SOCK=cef3da3af8b0de9b) Created new socket with FD=17 2020/12/21 09:09:50.364909 dev test MDW DBG connection (SOCK=cef3da3af8b0de9b) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.365730 dev test MDW INFO connection (SOCK=cef3da3af8b0de9b) Connected established 2020/12/21 09:09:50.365741 dev test MDW DBG connection (SOCK=cef3da3af8b0de9b) Successfully set TCP_NODELAY 2020/12/21 09:09:50.365744 dev test MDW DBG connection (SOCK=cef3da3af8b0de9b) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.365747 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23770,C=(nil); E=0x0 2020/12/21 09:09:50.365750 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f23680 SOCKET=0x5f23c70 2020/12/21 09:09:50.365753 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.365758 dev test MDW DBG ioctx (CTX=0x5f23b00,unknown) Pairing with SOCK=cef3da3af8b0de9b 2020/12/21 09:09:50.365784 dev test MDW DBG negotiation (CTX=0x5f23b00,sasl,SASLREQ=0x5f23040) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/cef3da3af8b0de9b"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.366792 dev test MDW DBG negotiation (CTX=0x5f23b00,sasl,SASLREQ=0x5f23040) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.367356 dev test MDW DBG negotiation (CTX=0x5f23b00,sasl,SASLREQ=0x5f23040) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.373254 dev test MDW WARN negotiation (CTX=0x5f23b00,sasl,SASLREQ=0x5f23040) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.373282 dev test MDW ERROR [0/] negotiation (CTX=0x5f23b00,sasl,SASLREQ=0x5f23040) Error: 0xce, SASL Step failed (ref: "fd3622e5-a45b-4908-7de5-90062ac11ea4") 2020/12/21 09:09:50.373287 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.373292 dev test MDW DBG retryq Adding PKT=0x5f2eaf0 to retry queue. retries=1, opaque=43, time=8434us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.373294 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.373297 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.373304 dev test MDW DBG ioctx (CTX=0x5f23b00,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.373340 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.373348 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.373352 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f016c0 (node3:11210) 2020/12/21 09:09:50.373362 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Creating new connection because none are available in the pool 2020/12/21 09:09:50.373368 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) New pool entry: I=0x5f23680 2020/12/21 09:09:50.373373 dev test MDW INFO connection (SOCK=c011107f95a99cb1) Starting. Timeout=100000us 2020/12/21 09:09:50.373383 dev test MDW DBG connection (SOCK=c011107f95a99cb1) Created new socket with FD=17 2020/12/21 09:09:50.373403 dev test MDW DBG connection (SOCK=c011107f95a99cb1) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.374212 dev test MDW INFO connection (SOCK=c011107f95a99cb1) Connected established 2020/12/21 09:09:50.374223 dev test MDW DBG connection (SOCK=c011107f95a99cb1) Successfully set TCP_NODELAY 2020/12/21 09:09:50.374226 dev test MDW DBG connection (SOCK=c011107f95a99cb1) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.374229 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Received result for I=0x5f23680,C=(nil); E=0x0 2020/12/21 09:09:50.374232 dev test MDW DBG lcbio_mgr (HE=0x5f1cca0) Assigning R=0x5f2e460 SOCKET=0x5f2e870 2020/12/21 09:09:50.374234 dev test MDW DBG server (SRV=0x5f016c0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.374239 dev test MDW DBG ioctx (CTX=0x5f22750,unknown) Pairing with SOCK=c011107f95a99cb1 2020/12/21 09:09:50.374264 dev test MDW DBG negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/c011107f95a99cb1"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.375322 dev test MDW DBG negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.375824 dev test MDW DBG negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.382041 dev test MDW WARN negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.382070 dev test MDW ERROR [0/] negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) Error: 0xce, SASL Step failed (ref: "b309a75d-5d6b-47fa-f8e8-f3751d5d652c") 2020/12/21 09:09:50.382076 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f016c0,IX=2) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.382081 dev test MDW DBG retryq Adding PKT=0x5f23080 to retry queue. retries=1, opaque=44, time=8723us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.382083 dev test MDW DBG retryq Next tick in 1 ms 2020/12/21 09:09:50.382087 dev test MDW DBG retryq Adding PKT=0x5f23180 to retry queue. retries=1, opaque=45, time=8728us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.382089 dev test MDW DBG retryq Next tick in 1 ms 2020/12/21 09:09:50.382096 dev test MDW DBG ioctx (CTX=0x5f22750,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.384188 dev test MDW DBG retryq Flush PKT=0x5f2eaf0 to network. retries=1, opaque=43, IX=1, time=19328us 2020/12/21 09:09:50.384202 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.384208 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f2e600 2020/12/21 09:09:50.384213 dev test MDW INFO connection (SOCK=a922085afc976335) Starting. Timeout=100000us 2020/12/21 09:09:50.384222 dev test MDW DBG connection (SOCK=a922085afc976335) Created new socket with FD=17 2020/12/21 09:09:50.384244 dev test MDW DBG connection (SOCK=a922085afc976335) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.384254 dev test MDW DBG retryq Next tick in 7 ms 2020/12/21 09:09:50.384884 dev test MDW INFO connection (SOCK=a922085afc976335) Connected established 2020/12/21 09:09:50.384894 dev test MDW DBG connection (SOCK=a922085afc976335) Successfully set TCP_NODELAY 2020/12/21 09:09:50.384897 dev test MDW DBG connection (SOCK=a922085afc976335) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.384900 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f2e600,C=(nil); E=0x0 2020/12/21 09:09:50.384903 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f2e460 SOCKET=0x5f2e870 2020/12/21 09:09:50.384905 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.384910 dev test MDW DBG ioctx (CTX=0x5f22750,unknown) Pairing with SOCK=a922085afc976335 2020/12/21 09:09:50.384937 dev test MDW DBG negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/a922085afc976335"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.385896 dev test MDW DBG negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.386417 dev test MDW DBG negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.392092 dev test MDW WARN negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.392124 dev test MDW ERROR [0/] negotiation (CTX=0x5f22750,sasl,SASLREQ=0x5f2e8e0) Error: 0xce, SASL Step failed (ref: "9628bcc5-f04d-4faa-a1e0-bb3275581731") 2020/12/21 09:09:50.392128 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.392135 dev test MDW DBG retryq Adding PKT=0x5fb7da0 to retry queue. retries=2, opaque=43, time=27277us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.392139 dev test MDW DBG retryq Next tick in 0 ms 2020/12/21 09:09:50.392147 dev test MDW DBG ioctx (CTX=0x5f22750,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.392174 dev test MDW DBG retryq Flush PKT=0x5f23080 to network. retries=1, opaque=44, IX=0, time=18816us 2020/12/21 09:09:50.392186 dev test MDW DBG retryq Flush PKT=0x5f23180 to network. retries=1, opaque=45, IX=0, time=18814us 2020/12/21 09:09:50.392194 dev test MDW DBG retryq Next tick in 19 ms 2020/12/21 09:09:50.393288 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.413370 dev test MDW DBG retryq Flush PKT=0x5fb7da0 to network. retries=2, opaque=43, IX=1, time=48510us 2020/12/21 09:09:50.413384 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.413390 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23700 2020/12/21 09:09:50.413405 dev test MDW INFO connection (SOCK=1f9b962cd7c92a68) Starting. Timeout=100000us 2020/12/21 09:09:50.413415 dev test MDW DBG connection (SOCK=1f9b962cd7c92a68) Created new socket with FD=17 2020/12/21 09:09:50.413437 dev test MDW DBG connection (SOCK=1f9b962cd7c92a68) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.414184 dev test MDW INFO connection (SOCK=1f9b962cd7c92a68) Connected established 2020/12/21 09:09:50.414194 dev test MDW DBG connection (SOCK=1f9b962cd7c92a68) Successfully set TCP_NODELAY 2020/12/21 09:09:50.414198 dev test MDW DBG connection (SOCK=1f9b962cd7c92a68) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.414200 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23700,C=(nil); E=0x0 2020/12/21 09:09:50.414203 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5f0bff0 SOCKET=0x5f22af0 2020/12/21 09:09:50.414206 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.414211 dev test MDW DBG ioctx (CTX=0x5f226c0,unknown) Pairing with SOCK=1f9b962cd7c92a68 2020/12/21 09:09:50.414237 dev test MDW DBG negotiation (CTX=0x5f226c0,sasl,SASLREQ=0x5f2e550) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/1f9b962cd7c92a68"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.415141 dev test MDW DBG negotiation (CTX=0x5f226c0,sasl,SASLREQ=0x5f2e550) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.415699 dev test MDW DBG negotiation (CTX=0x5f226c0,sasl,SASLREQ=0x5f2e550) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.421407 dev test MDW WARN negotiation (CTX=0x5f226c0,sasl,SASLREQ=0x5f2e550) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.421434 dev test MDW ERROR [0/] negotiation (CTX=0x5f226c0,sasl,SASLREQ=0x5f2e550) Error: 0xce, SASL Step failed (ref: "f9afd11f-07d8-4b63-f8d2-07da9fed3412") 2020/12/21 09:09:50.421439 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.421445 dev test MDW DBG retryq Adding PKT=0x5fb9ef0 to retry queue. retries=3, opaque=43, time=56587us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.421447 dev test MDW DBG retryq Next tick in 29 ms 2020/12/21 09:09:50.421450 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.421459 dev test MDW DBG ioctx (CTX=0x5f226c0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.421495 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.421503 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.421507 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f15800 (node1:11210) 2020/12/21 09:09:50.422506 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.452586 dev test MDW DBG retryq Flush PKT=0x5fb9ef0 to network. retries=3, opaque=43, IX=1, time=87726us 2020/12/21 09:09:50.452603 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.452608 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23700 2020/12/21 09:09:50.452613 dev test MDW INFO connection (SOCK=ede51f2079914a29) Starting. Timeout=100000us 2020/12/21 09:09:50.452623 dev test MDW DBG connection (SOCK=ede51f2079914a29) Created new socket with FD=17 2020/12/21 09:09:50.452653 dev test MDW DBG connection (SOCK=ede51f2079914a29) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.453448 dev test MDW INFO connection (SOCK=ede51f2079914a29) Connected established 2020/12/21 09:09:50.453459 dev test MDW DBG connection (SOCK=ede51f2079914a29) Successfully set TCP_NODELAY 2020/12/21 09:09:50.453462 dev test MDW DBG connection (SOCK=ede51f2079914a29) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.453465 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23700,C=(nil); E=0x0 2020/12/21 09:09:50.453468 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5fb9e60 SOCKET=0x5f2e5e0 2020/12/21 09:09:50.453470 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.453475 dev test MDW DBG ioctx (CTX=0x5f23ae0,unknown) Pairing with SOCK=ede51f2079914a29 2020/12/21 09:09:50.453501 dev test MDW DBG negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/ede51f2079914a29"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.454325 dev test MDW DBG negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.454847 dev test MDW DBG negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.460193 dev test MDW WARN negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.460222 dev test MDW ERROR [0/] negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) Error: 0xce, SASL Step failed (ref: "5b52f151-b01f-4567-e8ca-c412a2d6fd47") 2020/12/21 09:09:50.460226 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.460232 dev test MDW DBG retryq Adding PKT=0x5f2e6c0 to retry queue. retries=4, opaque=43, time=95375us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.460234 dev test MDW DBG retryq Next tick in 4 ms 2020/12/21 09:09:50.460238 dev test MDW DBG confmon Refreshing current cluster map (bucket: adeltour-br3) 2020/12/21 09:09:50.460246 dev test MDW DBG ioctx (CTX=0x5f23ae0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.460284 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.460293 dev test MDW DBG confmon Attempting to retrieve cluster map via CCCP 2020/12/21 09:09:50.460295 dev test MDW DBG cccp Re-Issuing CCCP Command on server struct 0x5f0def0 (node2:11210) 2020/12/21 09:09:50.460309 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Creating new connection because none are available in the pool 2020/12/21 09:09:50.460315 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) New pool entry: I=0x5f23700 2020/12/21 09:09:50.460322 dev test MDW INFO connection (SOCK=686919bd959a88c0) Starting. Timeout=100000us 2020/12/21 09:09:50.460331 dev test MDW DBG connection (SOCK=686919bd959a88c0) Created new socket with FD=17 2020/12/21 09:09:50.460353 dev test MDW DBG connection (SOCK=686919bd959a88c0) Scheduling I/O watcher for asynchronous connection completion. 2020/12/21 09:09:50.461139 dev test MDW INFO connection (SOCK=686919bd959a88c0) Connected established 2020/12/21 09:09:50.461152 dev test MDW DBG connection (SOCK=686919bd959a88c0) Successfully set TCP_NODELAY 2020/12/21 09:09:50.461155 dev test MDW DBG connection (SOCK=686919bd959a88c0) Successfully set TCP_KEEPALIVE 2020/12/21 09:09:50.461158 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Received result for I=0x5f23700,C=(nil); E=0x0 2020/12/21 09:09:50.461161 dev test MDW DBG lcbio_mgr (HE=0x5f42c00) Assigning R=0x5fb1cc0 SOCKET=0x5f23d00 2020/12/21 09:09:50.461163 dev test MDW DBG server (SRV=0x5f0def0) Session not yet negotiated. Negotiating 2020/12/21 09:09:50.461168 dev test MDW DBG ioctx (CTX=0x5f23ae0,unknown) Pairing with SOCK=686919bd959a88c0 2020/12/21 09:09:50.461194 dev test MDW DBG negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) HELO identificator: {"a":"libcouchbase/3.0.7 (Linux-3.10.0-1127.13.1.el7.x86_64; x86_64; GNU 4.9.4)","i":"46086c72045ac880/686919bd959a88c0"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x0f (Tracing), 0x0e (Unordered execution), 0x17 (Create as Deleted) 2020/12/21 09:09:50.462297 dev test MDW DBG negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) Server supports features: 0x03 (TCP nodelay), 0x04 (Mutation seqno), 0x06 (XATTR), 0x07 (XERROR), 0x08 (Select bucket), 0x0b (JSON), 0x0e (Unordered execution), 0x0f (Tracing) 2020/12/21 09:09:50.462834 dev test MDW DBG negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) Using SCRAM-SHA512 SASL mechanism 2020/12/21 09:09:50.468112 dev test MDW WARN retryq Failing command (pkt=0x5f2e6c0, opaque=43, retries=4, time=103250us, status=0x84) from retry queue: LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.468134 dev test MDW DBG TST Received response (Set) for key "ConnectionAndKeyValueCircuitBreakers_ChangePassword9" ; lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, global elapsed time: [103ms] 2020/12/21 09:09:50.468141 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.468146 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.468154 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 0 Event : 1 ************************************************** 2020/12/21 09:09:50.468156 dev test MDW DBG Failure Notifying failure 2020/12/21 09:09:50.468159 dev test MDW DBG onFailure Trying to move from state Closed to Open 2020/12/21 09:09:50.468163 dev test MDW INFO CircuitBreaker Failure threshold was reached for system [adeltour-br3]. Opening the circuit breaker. 2020/12/21 09:09:50.468167 dev test MDW DBG TST Server response status code: 132, Lcb error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error, kv error: 5 2020/12/21 09:09:50.468175 dev test MDW ERROR [5/kKVClientError_AuthenticationError] TST Error: LCB_ERR_AUTHENTICATION_FAILURE (206): Authentication error ; Failed to perform, as part of Set query, the operation for key [ConnectionAndKeyValueCircuitBreakers_ChangePassword9], on master server [node2:11210], global elapsed time: [103ms] 2020/12/21 09:09:50.468803 dev test MDW WARN negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) SASL auth failed with STATUS=0x20 2020/12/21 09:09:50.468839 dev test MDW ERROR [0/] negotiation (CTX=0x5f23ae0,sasl,SASLREQ=0x5f23860) Error: 0xce, SASL Step failed (ref: "9ae575a3-e998-4088-823c-d963b326fd9b") 2020/12/21 09:09:50.468849 dev test MDW ERROR [0/] server (CTX=(nil),,SRV=0x5f0def0,IX=1) Connection attempt failed. Received LCB_ERR_AUTHENTICATION_FAILURE (206) from libcouchbase, received 0 from operating system 2020/12/21 09:09:50.468855 dev test MDW DBG retryq Adding PKT=0x5f2e550 to retry queue. retries=1, opaque=47, time=8550us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.468857 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.468862 dev test MDW DBG retryq Adding PKT=0x5f2e6c0 to retry queue. retries=1, opaque=48, time=8554us, status=0x84, rc=LCB_ERR_AUTHENTICATION_FAILURE (206) 2020/12/21 09:09:50.468864 dev test MDW DBG retryq Next tick in 9 ms 2020/12/21 09:09:50.468872 dev test MDW DBG ioctx (CTX=0x5f23ae0,sasl) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1 2020/12/21 09:09:50.473962 dev test MDW DBG server (CTX=(nil),,SRV=0x5f016c0,IX=2) Scheduling next timeout for 100 ms. This is not an error 2020/12/21 09:09:50.479038 dev test MDW DBG retryq Flush PKT=0x5f2e550 to network. retries=1, opaque=47, IX=0, time=18732us 2020/12/21 09:09:50.479050 dev test MDW DBG retryq Flush PKT=0x5f2e6c0 to network. retries=1, opaque=48, IX=0, time=18730us 2020/12/21 09:09:50.480201 dev test MDW DBG confmon Not applying configuration received via CCCP (bucket="adeltour-br3", source=CCCP, address="node1"). No changes detected. A.rev=52291, B.rev=52291. Changes: servers=no, map=no, replicas=no 2020/12/21 09:09:50.480241 dev test MDW DBG Evaluate Checking state 2020/12/21 09:09:50.480252 dev test MDW DBG onEvaluate Trying to move from state Open to HalfOpen 2020/12/21 09:09:50.480258 dev test MDW DBG CircuitBreaker Error in FSM when processing event: ******************** FSM ERROR ******************** Err : #2 - kFSMErr_Callback ErrTxt : Callback returned an error At : /remote/tmp/rnd-css/bms_workflow_replica/mdw/Toolbox/15-0-0-15/include/toolbox/FSM.h #348 State : 2 Event : 2 ************************************************** 2020/12/21 09:09:50.480268 dev test MDW WARN [21/Failing command ] TST Error: LCB_ERR_GENERIC (100): Generic error code ; Circuit breaker is open. Cannot submit operation for key ['ConnectionAndKeyValueCircuitBreakers_ChangePassword'] (as part of Set operation) for key ["ConnectionAndKeyValueCircuitBreakers_ChangePassword"] 2020/12/21 09:09:50.480279 dev test MDW TST TST CurlHttpClient::CurlHttpClient start 2020/12/21 09:09:50.480297 dev test MDW TST TST CurlHttpClient::CurlHttpClient end 2020/12/21 09:09:50.480377 dev test MDW TST TST Libcurl: STATE: INIT => CONNECT handle 0x5f243d8; line 1657 (connection #-5000) 2020/12/21 09:09:50.480428 dev test MDW TST TST Libcurl: Added connection 0. The cache now contains 1 members 2020/12/21 09:09:50.480445 dev test MDW TST TST Libcurl: family0 == v4, family1 == v6 2020/12/21 09:09:50.480456 dev test MDW TST TST Libcurl: Trying node1:8091... 2020/12/21 09:09:50.480490 dev test MDW TST TST Libcurl: STATE: CONNECT => WAITCONNECT handle 0x5f243d8; line 1718 (connection #0) 2020/12/21 09:09:50.481287 dev test MDW TST TST Libcurl: Connected to node1 (node1) port 8091 (#0) 2020/12/21 09:09:50.481297 dev test MDW TST TST Libcurl: STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x5f243d8; line 1851 (connection #0) 2020/12/21 09:09:50.481302 dev test MDW TST TST Libcurl: Marked for [keep alive]: HTTP default 2020/12/21 09:09:50.481305 dev test MDW TST TST Libcurl: STATE: SENDPROTOCONNECT => DO handle 0x5f243d8; line 1872 (connection #0) 2020/12/21 09:09:50.481318 dev test MDW TST TST Libcurl: Server auth using Basic with user 'adeltour-br3' 2020/12/21 09:09:50.481350 dev test MDW TST TST Libcurl: POST /controller/changePassword HTTP/1.1 Host: node1:8091 Authorization: Basic YWRlbHRvdXItYnIzOndyb25ncGFzc3dvcmQ= User-Agent: kvclient_unittests Accept: */* Content-Type: application/x-www-form-urlencoded Content-Length: 16 Expect: 100-continue 2020/12/21 09:09:50.481361 dev test MDW TST TST Libcurl: STATE: DO => DO_DONE handle 0x5f243d8; line 1943 (connection #0) 2020/12/21 09:09:50.481365 dev test MDW TST TST Libcurl: STATE: DO_DONE => PERFORM handle 0x5f243d8; line 2064 (connection #0) 2020/12/21 09:09:50.483199 dev test MDW TST TST Libcurl: Mark bundle as not supporting multiuse 2020/12/21 09:09:50.483208 dev test MDW TST TST Libcurl: HTTP 1.1 or later with persistent connection 2020/12/21 09:09:50.483211 dev test MDW TST TST Libcurl: HTTP/1.1 100 Continue 2020/12/21 09:09:50.483257 dev test MDW TST TST Libcurl: password=testpwd 2020/12/21 09:09:50.483265 dev test MDW TST TST Libcurl: We are completely uploaded and fine 2020/12/21 09:09:50.499321 dev test MDW TST TST Libcurl: Mark bundle as not supporting multiuse 2020/12/21 09:09:50.499333 dev test MDW TST TST Libcurl: HTTP 1.1 or later with persistent connection 2020/12/21 09:09:50.499336 dev test MDW TST TST Libcurl: HTTP/1.1 200 OK 2020/12/21 09:09:50.499346 dev test MDW TST TST Libcurl: X-XSS-Protection: 1; mode=block 2020/12/21 09:09:50.499360 dev test MDW TST TST Libcurl: X-Permitted-Cross-Domain-Policies: none 2020/12/21 09:09:50.499372 dev test MDW TST TST Libcurl: X-Frame-Options: DENY 2020/12/21 09:09:50.499382 dev test MDW TST TST Libcurl: X-Content-Type-Options: nosniff 2020/12/21 09:09:50.499392 dev test MDW TST TST Libcurl: Server: Couchbase Server 2020/12/21 09:09:50.499408 dev test MDW TST TST Libcurl: Pragma: no-cache 2020/12/21 09:09:50.499419 dev test MDW TST TST Libcurl: Expires: Thu, 01 Jan 1970 00:00:00 GMT 2020/12/21 09:09:50.499432 dev test MDW TST TST Libcurl: Date: Mon, 21 Dec 2020 09:09:50 GMT 2020/12/21 09:09:50.499451 dev test MDW TST TST Libcurl: Content-Length: 0 2020/12/21 09:09:50.499462 dev test MDW TST TST Libcurl: Cache-Control: no-cache,no-store,must-revalidate 2020/12/21 09:09:50.499482 dev test MDW TST TST Libcurl: 2020/12/21 09:09:50.499492 dev test MDW TST TST Libcurl: STATE: PERFORM => DONE handle 0x5f243d8; line 2254 (connection #0) 2020/12/21 09:09:50.499494 dev test MDW TST TST Libcurl: multi_done 2020/12/21 09:09:50.499501 dev test MDW TST TST Libcurl: Connection #0 to host node1 left intact 2020/12/21 09:09:50.499505 dev test MDW TST TST Libcurl: Expire cleared (transfer 0x5f243d8) 2020/12/21 09:09:50.499516 dev test MDW TST TST CurlHttpClient::performRequest - No error 2020/12/21 09:09:50.499518 dev test MDW TST TST CurlHttpClient::performRequest - Got Response 2020/12/21 09:09:50.500155 dev test MDW ERROR [4/kKVClientError_ClientConfigurationError] StopChildProcess WaitPID: Child with PID 33993 (33993) exited with status 15 2020/12/21 09:09:50.500387 dev test MDW INFO TST Disconnecting from Couchbase 2020/12/21 09:09:50.500411 dev test MDW DBG server (CTX=0x5f0ca20,memcached,SRV=0x5f15800,IX=0) Finalizing context 2020/12/21 09:09:50.500419 dev test MDW DBG ioctx (CTX=0x5f0ca20,memcached) Destroying context. Pending Writes=0, Entered=false, Socket Refcount=1 2020/12/21 09:09:50.500588 dev test MDW INFO TST Disconnecting from Couchbase 2020/12/21 09:09:50.500671 dev test MDW ERROR [4/kKVClientError_ClientConfigurationError] StopChildProcess Kill: error 3 (No such process) occured while trying to kill process with pid33993 2020/12/21 09:09:50.500684 dev test MDW ERROR [4/kKVClientError_ClientConfigurationError] StopChildProcess WaitPID: error 10 (No child processes) occured while waiting for process with pid 33993 2020/12/21 09:09:50.500687 dev test MDW ERROR [4/kKVClientError_ClientConfigurationError] StopChildProcess WaitPID: Child with PID -1 (33993) exited with status 0