Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-1585

[SGW]TestSyncGatewayCreateLocalTLS fails with "context deadline exceeded: document count zero"

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.2
    • 2.1.0
    • None
    • 1

    Description

      Suite: P0

      Stack Trace:

      --- FAIL: TestOperator (856.43s)
      --- FAIL: TestOperator/TestSyncGatewayCreateLocalTLS (217.10s)
      crd_util.go:26: creating couchbase cluster: test-couchbase-x0rfg
      util.go:1304: context deadline exceeded: document count zero
      util.go:1305: goroutine 440 [running]:
      runtime/debug.Stack(0xc0006d3200, 0xc000905d20, 0x1)
      /jenkins/workspace/cbop-gke-p0-2.0.x/go/src/runtime/debug/stack.go:24 +0x9d
      github.com/couchbase/couchbase-operator/test/e2e/e2eutil.Die(0xc0006d3200, 0x19741c0, 0xc000592310)
      /jenkins/workspace/cbop-gke-p0-2.0.x/gopath/src/github.com/couchbase/couchbase-operator/test/e2e/e2eutil/util.go:1305 +0x77
      github.com/couchbase/couchbase-operator/test/e2e/e2eutil.MustVerifyDocCountInBucketNonZero(0xc0006d3200, 0xc000315700, 0xc000683200, 0x171c334, 0x7, 0xdf8475800)
      jenkins/workspace/cbop-gke-p0-2.0.x/gopath/src/github.com/couchbase/couchbase-operator/test/e2e/e2eutil/xdcr_util.go:150 +0x92
      github.com/couchbase/couchbase-operator/test/e2e.testSyncGatewayCreate(0xc0006d3200, 0xc000315700, 0xc000315700, 0x0, 0xc000476e60, 0x0)
      /jenkins/workspace/cbop-gke-p0-2.0.x/gopath/src/github.com/couchbase/couchbase-operator/test/e2e/sync-gateway.go:50 +0x208
      github.com/couchbase/couchbase-operator/test/e2e.TestSyncGatewayCreateLocalTLS(0xc0006d3200)
      /jenkins/workspace/cbop-gke-p0-2.0.x/gopath/src/github.com/couchbase/couchbase-operator/test/e2e/sync-gateway.go:67 +0x17c
      github.com/couchbase/couchbase-operator/test/e2e/framework.RecoverDecorator.func1(0xc0006d3200)
      /jenkins/workspace/cbop-gke-p0-2.0.x/gopath/src/github.com/couchbase/couchbase-operator/test/e2e/framework/test_util.go:347 +0x69
      testing.tRunner(0xc0006d3200, 0xc000439d40)
      /jenkins/workspace/cbop-gke-p0-2.0.x/go/src/testing/testing.go:909 +0xc9
      created by testing.(*T).Run
      /jenkins/workspace/cbop-gke-p0-2.0.x/go/src/testing/testing.go:960 +0x350

      Images Used:
      couchbase sync gateway: couchbase/sync-gateway:2.7.3-enterprise
      couchbase operator: couchbase/couchbase-operator-internal:2.0.2-110
      couchbase admission controller: couchbase/couchbase-admission-internal:2.0.2-110
      couchbase server: couchbase/server:6.5.1
      couchbase server upgrade: couchbase/server:6.5.1

      TestCase:  https://github.com/couchbase/couchbase-operator/blob/49172cbaab0f9ae227b86ac6cfe7cc62551a7988/test/e2e/sync-gateway.go#L63

      Job URL:  http://qa.sc.couchbase.com/view/Cloud/job/cbop-gke-p0-2.0.x/147/console

       The same behaviour is observed for these Sync Gateway test cases as well:

      TestSyncGatewayCreateLocalMutualTLS
      TestSyncGatewayCreateLocalMandatoryMutualTLS
      TestSyncGatewayCreateRemoteTLS
      TestSyncGatewayCreateRemoteMutualTLS
      TestSyncGatewayCreateRemoteMandatoryMutualTLS

      Job URL:  http://qa.sc.couchbase.com/view/Cloud/job/cbop-gke-p0-2.0.x/146/console

      Logs attached.(archive.tgz contains logs of TestSyncGatewayCreateLocalTLS failure, archive_SGW.tgz contains logs of all SGW test case failures)

      Attachments

        1. archive_SGW.tgz
          53.39 MB
        2. archive.tgz
          8.08 MB
        3. sg_debug.log
          9 kB
        4. sg_error.log
          0.2 kB
        5. sg_info.log
          8 kB
        6. sg_warn.log
          0.2 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          simon.murray Simon Murray added a comment - I'm not sure this is right, when not just link to the documentation https://docs.couchbase.com/operator/2.0/tutorial-tls.html#creating-a-couchbase-cluster-server-certificate

          Eric Schneider, Matt Ingenthron Simon Murray I agree with the suggestion of including release notes but to also point to the docs link that Simon has mentioned
          In notes .In addition, IIRC there was a docs page that discussed compatibility between different clients (XDCR, Sync Gatewat etc). Can we please add a footnote to that as well - people are likely going to miss this in release specific notes that are more point in time

          priya.rajagopal Priya Rajagopal added a comment - Eric Schneider , Matt Ingenthron Simon Murray I agree with the suggestion of including release notes but to also point to the docs link that Simon has mentioned In notes .In addition, IIRC there was a docs page that discussed compatibility between different clients (XDCR, Sync Gatewat etc). Can we please add a footnote to that as well - people are likely going to miss this in release specific notes that are more point in time

          agreed on the docs link, thanks for the clarification. I'll defer to Eric Schneider on how to handle the footnote. I took the liberty of opening a subtask so we don't forget the RN since this is closed.

          ingenthr Matt Ingenthron added a comment - agreed on the docs link, thanks for the clarification. I'll defer to Eric Schneider on how to handle the footnote. I took the liberty of opening a subtask so we don't forget the RN since this is closed.

          Thanks for the subtask Matt Ingenthron.

          priya.rajagopal Priya Rajagopal added a comment - Thanks for the subtask Matt Ingenthron .
          eric.schneider Eric Schneider (Inactive) added a comment - - edited

          Description for release notes:

          *Summary:* When running applications (such as Sync Gateway) that are using DNS SRV over TLS to connect to a Couchbase Cluster in the same Kubernetes cluster, lookup may fail hostname validation checks.
           
          *Workaround:* Add wildcard matches to the Subject Alternate Names (SANs) in the certificate for the Kubernetes-based host names.Specifically, add `DNS:*.$\{cluster}.$\{namespace}.svc` and `DNS:$\{cluster}-srv.$\{namespace}.svc` as discussed in the xref:tutorial-tls.adoc#creating-a-couchbase-cluster-server-certificate[Creating TLS Certificates] tutorial.

          eric.schneider Eric Schneider (Inactive) added a comment - - edited Description for release notes: *Summary:* When running applications (such as Sync Gateway) that are using DNS SRV over TLS to connect to a Couchbase Cluster in the same Kubernetes cluster, lookup may fail hostname validation checks.   *Workaround:* Add wildcard matches to the Subject Alternate Names (SANs) in the certificate for the Kubernetes-based host names.Specifically, add `DNS:*.$\{cluster}.$\{namespace}.svc` and `DNS:$\{cluster}-srv.$\{namespace}.svc` as discussed in the xref:tutorial-tls.adoc#creating-a-couchbase-cluster-server-certificate[Creating TLS Certificates] tutorial.

          People

            simon.murray Simon Murray
            prateek.kumar Prateek Kumar (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty