Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48888

XDCR - upgrade golang to 1.16 or later to address large heapIdle issues

    XMLWordPrintable

Details

    • Task
    • Resolution: Fixed
    • Major
    • 7.1.0
    • 7.1.0
    • XDCR
    • None
    • 1

    Description

      MB-48860 initially looks like XDCR was leaking memory and being killed (status 137).
      However, looking more closely, there's evidence that large amount of heap was not being freed:

      2021-10-11T19:26:55.729-07:00 INFO GOXDCR.ReplMgr: Mem stats = {"Alloc":7412721584,"TotalAlloc":666004936352,"Sys":15415353824,"Lookups":0,"Mallocs":5097536792,"Frees":5071878962,"HeapAlloc":7412721584,"HeapSys":14692745216,"HeapIdle":5389934592,"HeapInuse":9302810624,"HeapReleased":66060288
      		 
       
      		 
      "HeapInuse":9302810624,
      		 
      "HeapIdle":5389934592,
      		 
      HeapReleased":66060288

      Golang had an issue fixed recently where it now defaults to MADV_DONTNEED on Linux.
      https://github.com/golang/go/issues/33376

      This issue is fixed in golang 1.16 and XDCR should upgrade to take advantage of this, since currently it's sitting on 1.15.8, which is quite old.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MB-48838 [Magma, 10TB, KV+XDCR, 1%]: KV Rebalance-In failed during get_dcp_docs_estimate.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-48860 [System Test] Service 'goxdcr' exited with status 137. Execution timed out.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              neil.huang Neil Huang
              neil.huang Neil Huang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty