Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19883

indexer service exceeding quota

    XMLWordPrintable

Details

    Description

      Some analysis from John below. Seeing indexer being targeted by oom killer as it grows beyond quota of 15GB on a 30GB system. Effectively, indexer is killed and indexes need to rebuild.

      >>
      [6/8/16, 12:26:57 PM] john liang: From stats.log, I see indexer RSS grows to 30G (node has 31G RAM)
      [6/8/16, 12:26:59 PM] john liang:

      {<<"indexer/23819/mem_resident">>,29833822208}

      ,
      [6/8/16, 12:27:30 PM] john liang: This is at 06:49:39, a second before OOM kicks in
      [6/8/16, 12:27:31 PM] john liang: 2016-06-08T06:49:39.356-07:00,ns_1@172.23.105.60

      [6/8/16, 1:11:26 PM] john liang: For go runtime, it reports HeapInUse to be 5G
      [6/8/16, 1:11:38 PM] john liang: forestdb reports its memory used is 9.5G
      [6/8/16, 1:11:49 PM] john liang: So indexer reports memory used as 15G
      [6/8/16, 1:12:09 PM] john liang: But RSS says 30G
      [6/8/16, 1:12:39 PM] john liang: Golang also reports 5G of idle memory not freed yet
      [6/8/16, 1:12:56 PM] john liang: But I would still expect memory used to be around 20G

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              deepkaran.salooja Deepkaran Salooja
              tommie Tommie McAfee (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty