Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-57755

Detect and/or workaround systemd incorrect swappiness in RHEL8 & similar era distros

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 7.2.0
    • build
    • Linux with systemd using cgroups v1 (includes RHEL8, Ubuntu 20.04)
    • Ubuntu 64-bit
    • 0
    • No

    Description

      While investigating CBPS-1208, we found that despite setting vm.swappiness = 0, we were getting aggressive swap behaviour (with a big performance penalty) because all cgroups had their own swappiness setting, which was 60.

      There is a good RedHat article describing the problem here: https://access.redhat.com/solutions/6785021

      Any system with cgroups v1 (there are two main versions - 1 and 2) is susceptible to this, and it is quite painful to encounter.

      We hit this issue in perf team after upgrading our test servers from CentOS 7 to Ubuntu 20. It is reasonable to assume that customers may hit this issue in the future, and so it makes sense to investigate what we could do from a product POV to mitigate this problem.

      The issue is pretty easy to diagnose (there is an example in the RH article above) and there are workarounds such as switching to cgroups v2. One quick thing we could do is update our documentation to cover this.

      A more proactive solution could be to set the swappiness ourselves when running on systems with cgroups v1.

      Attachments

        Activity

          People

            ceej Chris Hillery
            Daniel.nagy Daniel Nagy
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:

              PagerDuty