Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2848

Attempt failover of missing Pod w/pvc during upgrade

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • .maintenance
    • 2.3.0
    • operator
    • None
    • 9 - Krakend, 10 - Krackeverlasting, 1 - Levianthanning, 2 - LeVIAthan or LeviaTHAN, 3 - Uk Sprinting, 4 - Who killed Tommie, 5 - No, 2.4.1 isn't released, 6- Yes, 2.4.1 is get released, 1 - 2.4.2 Dev Sprint, 2 - 2.5.0 (2 of 2), 3 - 2.5.0 - Oh No, 2.5.0 - The final countdown
    • 1

    Description

       

      If a Pod with a PersistentVolume is deleted between an upgrade cycle then the Operator complains that there is a volume without an associate PVC and throw error:

      {"level":"info","ts":1665013304.0493104,"logger":"cluster","msg":"Resource updated","cluster":"default/cb-example-couchbase-cluster","diff":"  (\n  \t\"\"\"\n  \t... // 30 identical lines\n  \t  status: \"True\"\n  \t  type: Balanced\n+ \t- lastTransitionTime: \"2022-10-05T23:41:44Z\"\n+ \t  lastUpdateTime: \"2022-10-05T23:41:44Z\"\n+ \t  message: 'required resource missing: pod for volume missing'\n+ \t  reason: ErrorEncountered\n+ \t  status: \"True\"\n+ \t  type: Error\n  \tcurrentVersion: 6.6.3\n  \tmembers:\n  \t... // 6 identical lines\n  \t\"\"\"\n  )\n"}
      {"level":"error","ts":1665013304.0725157,"logger":"controller","msg":"Failed to create Couchbase cluster","cluster":
      {"namespace":"default","name":"cb-example-couchbase-cluster"}
      ,"error":"required resource missing: pod for volume missing","stacktrace":"github.com/couchbase/couchbase-operator/pkg/controller.(*CouchbaseClusterReconciler).Reconcile\n\tgithub.com/couchbase/couchbase-operator/pkg/controller/controller.go:76

      (the stack trace is also not helpful)

       


       

       Under normal operations when a Pod is deleted we attempt an auto-failover and then at least rely on user to perform a manual failure.

      Attachments

        Activity

          People

            justin.ashworth Justin Ashworth
            tommie Tommie McAfee (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              PagerDuty