Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.0.0
-
Security Level: Public
-
Untriaged
-
Centos 64-bit
-
-
No
Description
Build
------
4.0.0-3310
Seen during xdcr + tpcc system test
2015-06-29 14:50:05 172.23.105.44:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.54:11210_0:Xmem is stuck
|
2015-06-29 14:49:31 172.23.105.48:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.58:11210_0:Xmem is stuck
|
2015-06-29 14:49:30 172.23.105.45:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.55:11210_1:Xmem is stuck
|
2015-06-29 14:49:29 172.23.105.50:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.61:11210_0:Xmem is stuck
|
2015-06-29 14:49:28 172.23.105.44:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.54:11210_0:Xmem is stuck
|
2015-06-29 14:49:27 172.23.105.52:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.63:11210_0:Xmem is stuck
|
2015-06-29 14:49:27 172.23.105.51:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.62:11210_0:Xmem is stuck
|
2015-06-29 14:49:27 172.23.105.49:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.60:11210_0:Xmem is stuck
|
2015-06-29 14:49:27 172.23.105.47:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.57:11210_0:Xmem is stuck
|
2015-06-29 14:35:41 172.23.105.52:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.63:11210_0:Xmem is stuck
|
|
|
2015-06-29 14:39:42 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
|
2015-06-29 14:38:44 172.23.105.45:xmem_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.55:11210_0:Xmem is stuck
|
2015-06-29 14:35:20 172.23.105.45:xmem_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.55:11210_1:Xmem is stuck
|
2015-06-29 14:31:27 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_1:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_1
|
2015-06-29 14:30:27 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
|
2015-06-29 14:29:30 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
|
2015-06-29 14:28:34 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
|
2015-06-29 14:27:36 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
|
2015-06-29 14:26:24 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
|
2015-06-29 14:26:14 172.23.105.47:xmem_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.57:11210_1:Xmem is stuck
|
2 Questions-
1. Error list is full on all 3 replications. Why are we seeing so many xmem and dcp stuck errors on all nodes?
2. Sometimes I see replication error list containing errors 30 mins old. Why did we not recover by then?
Attachments
For Gerrit Dashboard: MB-15494 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
52648,9 | MB-14999 -fix the followings A. setReadTimeout show up on this list (~5%) Call getConn for read less frequent, bigger readTimeout B. (*WrappedMCRequest).ConstructUniqueKey show up on the list (66.5M) C. reduce the number of call to convT2E by not using too much in event MB-15494 [system-test] GoXDCR: Many "dcp is stuck" and "xmem is stuck" errors seen in replications | master | goxdcr | Status: MERGED | +2 | +1 |