Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45954

Rebalance fails because of PrepareTopologyChange timeout

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      This issue is branched from MB-45869, please take logs from that ticket.

      Rebalance at 2021-04-21T04:04:51.677-07:00 fails because of PrepareTopologyChange timeout:

      2861252 [user:error,2021-04-21T08:41:40.426-07:00,ns_1@172.23.108.103:<0.23324.0>:ns_orchestrator:log_rebalance_completion:1404]Rebalance exited with reason {service_rebalance_failed,eventing,
      2861253                               {agent_died,<30247.24974.0>,
      2861254                                {linked_process_died,<30247.30085.1993>,
      2861255                                 {timeout,
      2861256                                  {gen_server,call,
      2861257                                   [<30247.25022.0>,
      2861258                                    {call,"ServiceAPI.PrepareTopologyChange",
      2861259                                     #Fun<json_rpc_connection.0.77329884>},
      2861260                                    60000]}}}}}.
      

      Which is a call to eventing on 172.23.123.27

      On 172.23.123.27 I see that eventing created a lot (~2000) of streaming connections against ns_server around that time, which I believe could lead to a timeout.

      Please investigate why there are so many streaming connections.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

             

            The fact that leaking connections lead to this issue was my just my guess. I think QE should rerun the original test and check CPU usage and connections.

            Arunkumar Senthilnathan

            Can you run the test that lead to this issue (the one from MB-45869)? Can you also collect logs from all eventing nodes when the test finishes (even if there are no errors)? I will take a look at those logs and close the ticket if there are no timeouts and the number of streaming connections is normal.

            timofey.barmin Timofey Barmin added a comment -   The fact that leaking connections lead to this issue was my just my guess. I think QE should rerun the original test and check CPU usage and connections. Arunkumar Senthilnathan Can you run the test that lead to this issue (the one from MB-45869 )? Can you also collect logs from all eventing nodes when the test finishes (even if there are no errors)? I will take a look at those logs and close the ticket if there are no timeouts and the number of streaming connections is normal.

            sure Timofey Barmin we have to wait for longevity to hit 3rd iteration - I will upload the logs once this happens with our latest longevity run

            arunkumar Arunkumar Senthilnathan (Inactive) added a comment - sure Timofey Barmin we have to wait for longevity to hit 3rd iteration - I will upload the logs once this happens with our latest longevity run

            Build couchbase-server-7.0.0-5131 contains eventing commit 6251420 with commit message:
            MB-45954: Fix race between closing channel and reading from socket

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-5131 contains eventing commit 6251420 with commit message: MB-45954 : Fix race between closing channel and reading from socket

            Timofey Barmin Our longevity against 7.0.0-5161 ran for 3 iterations - here are the logs collected at the end of 2nd and 3rd iterations:

            2nd iteration:
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.137.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.155.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.157.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.5.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.67.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.69.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.70.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.105.107.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.105.111.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.106.188.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.108.103.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.120.245.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.121.117.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.121.3.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.123.27.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.123.28.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.148.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.251.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.252.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.253.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.119.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.121.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.122.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.239.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.98.135.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.11.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.20.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.21.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.25.zip

            3rd iteration:
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.155.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.157.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.5.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.69.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.70.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.105.107.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.105.111.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.106.100.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.106.188.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.108.103.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.120.245.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.121.117.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.121.3.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.123.27.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.123.28.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.148.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.251.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.252.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.253.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.119.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.121.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.122.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.239.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.98.135.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.11.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.20.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.21.zip
            url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.25.zip

            Cluster config:
            fts : 2 ===== > [172.23.104.155:8091 172.23.96.148:8091]
            cbas : 3 ===== > [172.23.104.157:8091 172.23.104.67:8091 172.23.106.188:8091]
            eventing : 3 ===== > [172.23.104.5:8091 172.23.123.27:8091 172.23.98.135:8091] kv : 12 ===== > [172.23.104.69:8091 172.23.104.70:8091 172.23.106.100:8091 172.23.108.103:8091 172.23.121.3:8091 172.23.97.119:8091 172.23.97.121:8091 172.23.97.122:8091 172.23.97.239:8091 172.23.99.20:8091 172.23.99.21:8091 172.23.99.25:8091]
            n1ql : 2 ===== > [172.23.120.245:8091 172.23.96.251:8091]
            backup : 1 ===== > [172.23.123.28:8091]
            index : 7 ===== > [172.23.104.137:8091 172.23.105.107:8091 172.23.105.111:8091 172.23.121.117:8091 172.23.96.252:8091 172.23.96.253:8091 172.23.99.11:8091]

            Please take a look at the logs and see if this issue can be closed

            arunkumar Arunkumar Senthilnathan (Inactive) added a comment - - edited Timofey Barmin Our longevity against 7.0.0-5161 ran for 3 iterations - here are the logs collected at the end of 2nd and 3rd iterations: 2nd iteration: url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.137.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.155.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.157.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.5.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.67.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.69.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.104.70.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.105.107.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.105.111.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.106.188.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.108.103.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.120.245.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.121.117.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.121.3.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.123.27.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.123.28.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.148.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.251.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.252.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.96.253.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.119.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.121.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.122.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.97.239.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.98.135.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.11.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.20.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.21.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621230447/collectinfo-2021-05-17T054729-ns_1%40172.23.99.25.zip 3rd iteration: url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.155.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.157.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.5.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.69.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.104.70.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.105.107.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.105.111.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.106.100.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.106.188.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.108.103.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.120.245.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.121.117.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.121.3.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.123.27.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.123.28.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.148.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.251.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.252.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.96.253.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.119.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.121.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.122.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.97.239.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.98.135.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.11.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.20.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.21.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621290512/collectinfo-2021-05-17T222834-ns_1%40172.23.99.25.zip Cluster config: fts : 2 ===== > [172.23.104.155:8091 172.23.96.148:8091] cbas : 3 ===== > [172.23.104.157:8091 172.23.104.67:8091 172.23.106.188:8091] eventing : 3 ===== > [172.23.104.5:8091 172.23.123.27:8091 172.23.98.135:8091] kv : 12 ===== > [172.23.104.69:8091 172.23.104.70:8091 172.23.106.100:8091 172.23.108.103:8091 172.23.121.3:8091 172.23.97.119:8091 172.23.97.121:8091 172.23.97.122:8091 172.23.97.239:8091 172.23.99.20:8091 172.23.99.21:8091 172.23.99.25:8091] n1ql : 2 ===== > [172.23.120.245:8091 172.23.96.251:8091] backup : 1 ===== > [172.23.123.28:8091] index : 7 ===== > [172.23.104.137:8091 172.23.105.107:8091 172.23.105.111:8091 172.23.121.117:8091 172.23.96.252:8091 172.23.96.253:8091 172.23.99.11:8091] Please take a look at the logs and see if this issue can be closed
            timofey.barmin Timofey Barmin added a comment - - edited

            Thanks Arunkumar Senthilnathan
            Number of rest calls looks good for all three eventing nodes:


            timofey.barmin Timofey Barmin added a comment - - edited Thanks Arunkumar Senthilnathan Number of rest calls looks good for all three eventing nodes:

            People

              arunkumar Arunkumar Senthilnathan (Inactive)
              timofey.barmin Timofey Barmin
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty