Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2485

Check that OpenShift podPidsLimit / pidsLimit is not default 1024

    XMLWordPrintable

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • not-targeted
    • operator
    • None
    • 1

    Description

      As per https://access.redhat.com/solutions/5366631, OpenShift by default only allows 1024 processes (also threads) to be created - see https://access.redhat.com/solutions/5366631

      This can cause significant problems for Couchbase Server as it typically needs more threads than that (say on a 16 core machine) - see linked CBSEs.

      As I understand it, the Operator cannot directly check the limit; however I believe that the limit is exposed inside the container via /sys - specifically /sys/fs/cgroup/pids/pids.max:

      Request is to check the value of this, and fail certification if the value is less than some more sensible number - 10,000 seems reasonable given that's what we currently check for the nproc ulimit:

      [Thu 21 Oct 16:20:58 BST 2021] simon@symphony ~/src/github.com/couchbase/couchbase-operator/dist/couchbase-autonomous-operator-kubernetes_2.3.0-beta1-linux-x86_64 bin/cao certify --image couchbase/operator-certification:2.3.0-beta1 -- -server-image couchbase/server:7.0.2 -test TestCreateCluster
      Initializing ...
      Creating service account ...
      Creating cluster role ...
      Creating cluster role binding ...
      Creating artifacts volume ...
      Creating pull secrets ...
      Creating certification pod ...
      Waiting for certification pod to become ready ...
      Certification pod running, streaming logs ...
      couchbase-operator-certification 2.3.0 (build 999)
      INFO[0000] Platform Preflight Checks            
      INFO[0000] Number of processes = unlimited (>= 10000) ✔ 
      INFO[0000] Number of open files = 1048576 (>= 70000) ✔ 
      

      NOTE: I don't have an OpenShift environment to hand, so I cannot be sure if /sys/fs/cgroup/pids/pids.max is the correct path (or even exists there) - that does exist under Docker, and is a reasonable sibling path to the /sys/fs/cgroup/pids/pids.current path mentioned in the aforementioned RH Knowledgebase article.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            There are no commits yet on this issue.

            People

              simon.murray Simon Murray
              drigby Dave Rigby
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty