Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-34643

DCP replication does not recover after SIGKILL/restart to memcached (dupe PREPARE?) [ETA 2019/7/19]

    XMLWordPrintable

Details

    Description

      Build: 6.5.0-3527

      Scenario:

      1. Two node cluster, single Couchbase bucket (replica=1)
      2. Start doc_ops
      3. While sync_write creates are running, kill Memcached on node-1 (kill -9)

      Observations:

      Memcached restarts on node-1, all doc_ops are failing without any valid response from the server. In the packet dump, able to see the response status as "Unknown (0x00a3)". No exception was thrown from SDK end as well.

      Expected behavior:

      Once Memcached restarts, doc_ops should be handled without any errors. Even in case of some errors, SDK should report an exception to the user.

      PCAP with Unknown status message:

      TAF test case:

      crash_test.crash_process.CrashTest.test_crash_process,nodes_init=2,replicas=1,num_items=10000,process=memcached,service=memcached,sig_type=sigkill,target_node=replica,durability=MAJORITY

       

      Also attaching test.log and unknown_response_ops_failed.pcapng

       

      Attachments

        1. image.png
          image.png
          180 kB
        2. image-2019-06-18-18-27-46-134.png
          image-2019-06-18-18-27-46-134.png
          181 kB
        3. Screen Shot 2019-06-18 at 14.32.08.png
          Screen Shot 2019-06-18 at 14.32.08.png
          441 kB
        4. test_on_3770.log
          1.89 MB
        5. test.log
          9 kB
        6. unknown_response_ops_failed.pcapng
          735 kB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ashwin.govindarajulu Ashwin Govindarajulu
              ashwin.govindarajulu Ashwin Govindarajulu
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty