Uploaded image for project: 'Couchbase Monitoring and Observability Stack'
  1. Couchbase Monitoring and Observability Stack
  2. CMOS-411

Check for reverse lookup to avoid prometheus load config delays

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • Major
    • 0.2
    • None
    • None
    • None

    Description

      It has been found that lack of reverse lookup can affect promethues reload delays leading to metrics blank out in 7.0 and later.

       Large load time can be found from logs similar to below:

      grep  "Completed loading of configuration file" ns_server.prometheus.log | cut -d' ' -f2,4,5,6,8,10
      		 
      ts=2022-03-11T19:13:34.382Z msg="Completed loading of file" totalDuration=33.435783142s
      		 
      ts=2022-03-11T19:18:30.480Z msg="Completed loading of file" totalDuration=40.03937375s
      		 
      ts=2022-03-11T19:20:44.633Z msg="Completed loading of file" totalDuration=40.038709051s
      		 
      ts=2022-03-11T19:34:03.886Z msg="Completed loading of file" totalDuration=25.871754809s
      		 
      ts=2022-03-11T19:44:16.972Z msg="Completed loading of file" totalDuration=40.039851597s
      		 
      ts=2022-03-11T19:44:57.017Z msg="Completed loading of file" totalDuration=40.044304865s
      		 
      ts=2022-03-11T19:45:37.058Z msg="Completed loading of file" totalDuration=40.041222275s
      		 
      ts=2022-03-11T19:46:17.095Z msg="Completed loading of file" totalDuration=40.03701188s
      		 
      ts=2022-03-11T19:46:57.132Z msg="Completed loading of file" totalDuration=40.037303612s
      		 
      ts=2022-03-11T19:47:37.175Z msg="Completed loading of file" totalDuration=40.042158873s
      		 
      ts=2022-03-11T19:48:17.213Z msg="Completed loading of file" totalDuration=40.038309107s
      		 
      ts=2022-03-11T19:48:57.251Z msg="Completed loading of file" totalDuration=40.038229631s
      		 
      ts=2022-03-11T19:49:37.293Z msg="Completed loading of file" totalDuration=40.041488195s
      		 
      ts=2022-03-11T19:50:17.332Z msg="Completed loading of file" totalDuration=40.03921885s
      		 
      ts=2022-03-11T19:50:57.370Z msg="Completed loading of file" totalDuration=40.037955946s
      		 
      ts=2022-03-11T19:51:37.418Z msg="Completed loading of file" totalDuration=40.048200152s
      		 
      ts=2022-03-11T19:52:17.458Z msg="Completed loading of file" totalDuration=40.040270134s
      		 
      ts=2022-03-11T19:52:57.497Z msg="Completed loading of file" totalDuration=40.038349768s
      		 
      ts=2022-03-11T19:53:37.535Z msg="Completed loading of file" totalDuration=40.038080412s

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jishnu.thankamani Jishnu Thankamani
            jishnu.thankamani Jishnu Thankamani
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty