Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37811

[CX] Analytics cluster down on IP reassignment

    XMLWordPrintable

Details

    • Untriaged
    • Unknown
    • CX Sprint 185

    Description

      As observed in a customer Kubernetes environment, the CC node was recreated with the existing storage, but assigned a new IP address. Due to network address caching, one of the existing NCs was perpetually unable to connect to the CC as it did not realize the IP address update until the driver could be restarted.

      We should set networkaddress.cache.ttl to a reasonable value, so that we are protected from this situation. e.g. something <= 5 minutes

       

      The above suggestion is insufficient, due to caching above the name service which is also a factor.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Arunkumar Senthilnathan can you pls take a look at this - it is K8S related.

            mihir.kamdar Mihir Kamdar (Inactive) added a comment - Arunkumar Senthilnathan can you pls take a look at this - it is K8S related.

            Covered by regression tests written by devs

            arunkumar Arunkumar Senthilnathan (Inactive) added a comment - Covered by regression tests written by devs

            Build couchbase-server-1006.5.1-1125 contains cbas commit c5f4aa7 with commit message:
            MB-37811: configure InetAddress cache ttl to 15s

            build-team Couchbase Build Team added a comment - Build couchbase-server-1006.5.1-1125 contains cbas commit c5f4aa7 with commit message: MB-37811 : configure InetAddress cache ttl to 15s

            Build couchbase-server-1006.5.1-1125 contains cbas-core commit 9cb70ef with commit message:
            MB-37811: remove restart node workarounds

            build-team Couchbase Build Team added a comment - Build couchbase-server-1006.5.1-1125 contains cbas-core commit 9cb70ef with commit message: MB-37811 : remove restart node workarounds

            Build couchbase-server-6.6.2-9599 contains cbas-core commit 9cb70ef with commit message:
            MB-37811: remove restart node workarounds

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9599 contains cbas-core commit 9cb70ef with commit message: MB-37811 : remove restart node workarounds

            People

              arunkumar Arunkumar Senthilnathan (Inactive)
              michael.blow Michael Blow
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty