Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49794

SystemEventLogs: Unable to fetch events after getting into 'unable to failover nodes' state

    XMLWordPrintable

Details

    Description

      Build: 7.1.0 build 1787

      Not able to fetch any events after hitting "Unable to automatically failover nodes" state

      Steps:

      1. Four node cluster
      2. Couchbase bucket with replica=0
      3. Stop memcached on one node (here 172.23.105.212)
      4. Wait for auto failover threshold to reach, which results in,

        Could not automatically fail over nodes (['ns_1@172.23.105.212']). Would lose vbuckets in the following buckets: ["default"]

      Observation:

      After hitting this state, we are unable to retrieve any events happened after this.

      Note: Events happened prior to this is still accessible.

      Curl outputs:

          {
            "timestamp": "2021-11-27T02:36:32.758Z",
            "event_id": 8202,
            "component": "data",
            "description": "Memcached configuration changed",
            "severity": "info",
            "node": "172.23.105.155",
            "uuid": "efed0fe7-9a2e-42f6-b58a-0e3a8f7624a4",
            "extra_attributes": {
              "old_settings": {},
              "new_settings": {
                "num_reader_threads": "default",
                "num_writer_threads": "default",
                "num_storage_threads": "default"
              }
            }
          }
        ]
      }
       
      $ curl -X GET -u Administrator:password 172.23.105.155:8091/events?sinceTime=2021-11-27T02:36:32.758Z\&limit=-1 | jq .
      [
        "Unexpected server error, request logged."
      ]
       
      $ curl -X GET -u Administrator:password 172.23.105.155:8091/events?sinceTime=2021-11-27T02:36:32.758Z | jq .
      [
        "Unexpected server error, request logged."
      ]
       
      $ curl -X GET -u Administrator:password 172.23.105.155:8091/events?limit=-1 | jq .
      [
        "Unexpected server error, request logged."
      ]
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ashwin.govindarajulu Ashwin Govindarajulu
            ashwin.govindarajulu Ashwin Govindarajulu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty