Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-544

events being returned in different order than operator reports

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • 1.0.0
    • None
    • None

    Description

      Is some tests we kill 3 nodes and wait for down events for each of the three nodes in parallel, returning the order in which the events are seen. However, the operator will report a different ordering. In this example, nodes 2, 5, and 8 are killed in that order and wait for gorountine is spun up for each of these nodes, listening for down event. The goroutines report the order as 2, 8, then 5, but then the operator says the order was 2, 5, 8:

          util.go:187: Expected events to be:
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0000 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0001 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0002 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0003 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0004 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0005 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0006 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0007 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0008 added to cluster
       
          Type: Normal | Reason: RebalanceStarted | Message: A rebalance has been started to balance data across the cluster
       
          Type: Normal | Reason: RebalanceCompleted | Message: A rebalance has completed
       
          Type: Normal | Reason: BucketCreated | Message: A new bucket `testBucket` was created
       
          Type: Warning | Reason: MemberDown | Message: Existing member test-couchbase-v9fxp-0002 down
       
          Type: Warning | Reason: MemberDown | Message: Existing member test-couchbase-v9fxp-0008 down
       
          Type: Warning | Reason: MemberDown | Message: Existing member test-couchbase-v9fxp-0005 down
       
          Type: Warning | Reason: MemberFailedOver | Message: Existing member test-couchbase-v9fxp-0008 failed over
       
          Type: Warning | Reason: MemberFailedOver | Message: Existing member test-couchbase-v9fxp-0002 failed over
       
          Type: Warning | Reason: MemberFailedOver | Message: Existing member test-couchbase-v9fxp-0005 failed over
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0009 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0010 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0011 added to cluster
       
          Type: Normal | Reason: RebalanceStarted | Message: A rebalance has been started to balance data across the cluster
       
          Type: Normal | Reason: MemberRemoved | Message: Existing member test-couchbase-v9fxp-0002 removed from the cluster
       
          Type: Normal | Reason: MemberRemoved | Message: Existing member test-couchbase-v9fxp-0005 removed from the cluster
       
          Type: Normal | Reason: MemberRemoved | Message: Existing member test-couchbase-v9fxp-0008 removed from the cluster
       
          Type: Normal | Reason: RebalanceCompleted | Message: A rebalance has completed
       
          
       
          but got:
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0000 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0001 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0002 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0003 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0004 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0005 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0006 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0007 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0008 added to cluster
       
          Type: Normal | Reason: RebalanceStarted | Message: A rebalance has been started to balance data across the cluster
       
          Type: Normal | Reason: RebalanceCompleted | Message: A rebalance has completed
       
          Type: Normal | Reason: BucketCreated | Message: A new bucket `testBucket` was created
       
          Type: Warning | Reason: MemberDown | Message: Existing member test-couchbase-v9fxp-0002 down
       
          Type: Warning | Reason: MemberDown | Message: Existing member test-couchbase-v9fxp-0005 down
       
          Type: Warning | Reason: MemberDown | Message: Existing member test-couchbase-v9fxp-0008 down
       
          Type: Warning | Reason: MemberFailedOver | Message: Existing member test-couchbase-v9fxp-0002 failed over
       
          Type: Warning | Reason: MemberFailedOver | Message: Existing member test-couchbase-v9fxp-0005 failed over
       
          Type: Warning | Reason: MemberFailedOver | Message: Existing member test-couchbase-v9fxp-0008 failed over
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0009 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0010 added to cluster
       
          Type: Normal | Reason: NewMemberAdded | Message: New member test-couchbase-v9fxp-0011 added to cluster
       
          Type: Normal | Reason: RebalanceStarted | Message: A rebalance has been started to balance data across the cluster
       
          Type: Normal | Reason: MemberRemoved | Message: Existing member test-couchbase-v9fxp-0002 removed from the cluster
       
          Type: Normal | Reason: MemberRemoved | Message: Existing member test-couchbase-v9fxp-0005 removed from the cluster
       
          Type: Normal | Reason: MemberRemoved | Message: Existing member test-couchbase-v9fxp-0008 removed from the cluster
       
          Type: Normal | Reason: RebalanceCompleted | Message: A rebalance has completed
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            korrigan.clark Korrigan Clark created issue -
            mikew Mike Wiederhold [X] (Inactive) made changes -
            Field Original Value New Value
            Assignee Mike Wiederhold [ mikew ] Korrigan Clark [ korrigan.clark ]
            mikew Mike Wiederhold [X] (Inactive) made changes -
            Fix Version/s 1.0.0 [ 15036 ]
            mikew Mike Wiederhold [X] (Inactive) made changes -
            Comment [ I need logs for this issue as well as the name of the test case because it's not clear that this is an operator issue. ]
            mikew Mike Wiederhold [X] (Inactive) made changes -
            Link This issue duplicates K8S-531 [ K8S-531 ]
            mikew Mike Wiederhold [X] (Inactive) made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Resolved [ 5 ]

            Closing since this is tracked in K8S-531

            ashwin.govindarajulu Ashwin Govindarajulu added a comment - Closing since this is tracked in K8S-531
            ashwin.govindarajulu Ashwin Govindarajulu made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

            People

              korrigan.clark Korrigan Clark
              korrigan.clark Korrigan Clark
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty