Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-15494

[system-test] GoXDCR: Many "dcp is stuck" and "xmem is stuck" errors seen in replications

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 4.0.0
    • 4.0.0
    • XDCR
    • Security Level: Public

    Description

      Build
      ------
      4.0.0-3310

      Seen during xdcr + tpcc system test

      2015-06-29 14:50:05 172.23.105.44:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.54:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:31 172.23.105.48:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.58:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:30 172.23.105.45:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.55:11210_1:Xmem is stuck
      		 
      2015-06-29 14:49:29 172.23.105.50:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.61:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:28 172.23.105.44:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.54:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:27 172.23.105.52:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.63:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:27 172.23.105.51:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.62:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:27 172.23.105.49:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.60:11210_0:Xmem is stuck
      		 
      2015-06-29 14:49:27 172.23.105.47:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.57:11210_0:Xmem is stuck
      		 
      2015-06-29 14:35:41 172.23.105.52:xmem_2c182d4d40b6acb63caff554807edbab/ORDER_LINE/ORDER_LINE_172.23.105.63:11210_0:Xmem is stuck
      		 
       
      		 
       
      		 
      2015-06-29 14:39:42 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
      		 
      2015-06-29 14:38:44 172.23.105.45:xmem_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.55:11210_0:Xmem is stuck
      		 
      2015-06-29 14:35:20 172.23.105.45:xmem_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.55:11210_1:Xmem is stuck
      		 
      2015-06-29 14:31:27 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_1:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_1
      		 
      2015-06-29 14:30:27 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
      		 
      2015-06-29 14:29:30 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
      		 
      2015-06-29 14:28:34 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
      		 
      2015-06-29 14:27:36 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
      		 
      2015-06-29 14:26:24 172.23.105.45:dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0:Dcp is stuck for dcp nozzle dcp_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.45:11210_0
      		 
      2015-06-29 14:26:14 172.23.105.47:xmem_2c182d4d40b6acb63caff554807edbab/CUSTOMER/CUSTOMER_172.23.105.57:11210_1:Xmem is stuck

      2 Questions-

      1. Error list is full on all 3 replications. Why are we seeing so many xmem and dcp stuck errors on all nodes?

      2. Sometimes I see replication error list containing errors 30 mins old. Why did we not recover by then?

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            apiravi Aruna Piravi (Inactive)
            apiravi Aruna Piravi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty