Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47254

[CBSE] Index Logs: Dedupe repeated log messages

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      The logs for one index node in the cluster only cover a 1 hour period, due to them being spammed with the following repeatedtly:

      2021-07-06T20:55:05.754-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.755-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused
      		 
      2021-07-06T20:55:05.756-04:00 [Error] WatcherServer.runOnce() error : dial tcp XXX:9100: getsockopt: connection refused

      This makes the logs all but useless when it comes to investigating issues. It would be better to instead have some kind of deduplication given the frequency, or better tuning of the logging to avoid this situation.

      Attachments

        For Gerrit Dashboard: MB-47254
        # Subject Branch Project Status CR V
        167799,4 MB-47254 (7.1.0 1910) Avoid log flooding from watcherServer connect fail unstable indexing Status: MERGED +2 +1
        167800,7 MB-47254 (7.1.0 1910) Avoid log flooding from watcherServer connect fail master gometa Status: MERGED +2 +1

        Activity

          People

            hemant.rajput Hemant Rajput
            tim.bradgate Tim Bradgate (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty