Uploaded image for project: 'Couchbase Kafka Connector'
  1. Couchbase Kafka Connector
  2. KAFKAC-172

Kafka Connector doesn't work with Istio

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-dp.1
    • None
    • 1

    Description

      We see 400 bad responses and connection issues when Kafka Connector is deployed with service mesh (Istio) enabled, but the Couchbase Server pods themselves have Istio disabled. 

      Also, as per our autonomous operator documentation on service meshes in Kubernetes Networking:

      The use of the Operator with service meshes (such as Istio) is not supported and will not work. The Operator and Couchbase cluster must be deployed in a namespace with the service mesh disabled in Kubernetes clusters where these technologies are deployed.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            matt.carabine Matt Carabine added a comment - - edited

            To be clear here, the request is not necessarily to 'fix' the Kafka Connector to work with Istio if this is not possible.

            This issue is more tracking that it doesn't seem to work with Istio and to either test + fix it (suspect maybe the issue needs fixing on the Couchbase side) or to document this somewhere.
            In the initial pass this KAFKAC is some level of 'documentation' indicating that it doesn't work.

            matt.carabine Matt Carabine added a comment - - edited To be clear here, the request is not necessarily to 'fix' the Kafka Connector to work with Istio if this is not possible. This issue is more tracking that it doesn't seem to work with Istio and to either test + fix it (suspect maybe the issue needs fixing on the Couchbase side) or to document this somewhere. In the initial pass this KAFKAC is some level of 'documentation' indicating that it doesn't work.

            Hm. Should we also document that it doesn't work on ARM or PowerPC? Or that it won't work on an iPhone?

            I'm not sure that documenting service meshes as explicitly not supported. Setting up Istio may be possible, but that's a function of setting up Istio for a generic TCP service, which is not something I think someone would reasonably expect Couchbase to document.

            So turning it around, I think if we later test/document it, that's great. If it's not documented and the user decides to use Istio, that's an exercise for the user.

            Okay, my initial examples a bit silly, but you could think of it as whether or not it runs across IPSec/VPN or the like. Setting up the network/routing/name resolution is not a function of the Connector, so the Connector typically wouldn't document it.

            ingenthr Matt Ingenthron added a comment - Hm. Should we also document that it doesn't work on ARM or PowerPC? Or that it won't work on an iPhone? I'm not sure that documenting service meshes as explicitly not supported. Setting up Istio may be possible, but that's a function of setting up Istio for a generic TCP service, which is not something I think someone would reasonably expect Couchbase to document. So turning it around, I think if we later test/document it, that's great. If it's not documented and the user decides to use Istio, that's an exercise for the user. Okay, my initial examples a bit silly, but you could think of it as whether or not it runs across IPSec/VPN or the like. Setting up the network/routing/name resolution is not a function of the Connector, so the Connector typically wouldn't document it.

            To turn that round, why do we document that the Operator doesn't work with Istio?
            Why do we consider Lambda to be any different to any other platform for SDKs etc and think it needs to be tested + documented independently?

            I don't necessarily disagree with what you're saying, but the boundaries we sometimes choose seem pretty arbitrary IMHO.

            matt.carabine Matt Carabine added a comment - To turn that round, why do we document that the Operator doesn't work with Istio? Why do we consider Lambda to be any different to any other platform for SDKs etc and think it needs to be tested + documented independently? I don't necessarily disagree with what you're saying, but the boundaries we sometimes choose seem pretty arbitrary IMHO.

            Also, as mentioned, in a sense this KAFKAC is documentation for a user coming in having problems to go "Oh, this doesn't work, cool okay". Maybe we never need to put anything official in our actual docs themselves

            matt.carabine Matt Carabine added a comment - Also, as mentioned, in a sense this KAFKAC is documentation for a user coming in having problems to go "Oh, this doesn't work, cool okay". Maybe we never need to put anything official in our actual docs themselves

            It's not really on topic for this issue, but…

            At least AWS Lambda is explicitly different because they freeze/thaw processes. They basically do not allow us to run background threads. We run background threads to update cluster topology (among other things).

            And yes, it might be a bit arbitrary, but to the end user, running a library in a runtime would reasonably be expected to work out of the box. We just need to point out that the runtime they've selected is definitely different and not fundamentally supportable without changes. AWS Lambda is not like the IBM JDK or Oracle JRockit. It's a specific runtime with specific security constraints and specifically breaks things one would reasonably assume to work, like a background thread will keep running.

            ingenthr Matt Ingenthron added a comment - It's not really on topic for this issue, but… At least AWS Lambda is explicitly different because they freeze/thaw processes. They basically do not allow us to run background threads. We run background threads to update cluster topology (among other things). And yes, it might be a bit arbitrary, but to the end user, running a library in a runtime would reasonably be expected to work out of the box. We just need to point out that the runtime they've selected is definitely different and not fundamentally supportable without changes. AWS Lambda is not like the IBM JDK or Oracle JRockit. It's a specific runtime with specific security constraints and specifically breaks things one would reasonably assume to work, like a background thread will keep running.
            david.nault David Nault added a comment -

            When we have a compatibility matrix, we can add a note about the Istio incompatibilty

            david.nault David Nault added a comment - When we have a compatibility matrix, we can add a note about the Istio incompatibilty
            david.nault David Nault added a comment -

            Mentioned Istio as a "Known Incopatibility" in the 4.0 compatibility matrix

            david.nault David Nault added a comment - Mentioned Istio as a "Known Incopatibility" in the 4.0 compatibility matrix

            People

              david.nault David Nault
              yuvraj.kanwar Yuvraj Kanwar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty