Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49004

Performance : Deployment hung

    XMLWordPrintable

Details

    Description

      Build: 7.1.0 build 1517 , not seen on 1472

      Test job: http://perf.jenkins.couchbase.com/job/themis/13189/console

      Error in logs

      2021-10-19T03:09:05.825-07:00 [Warn] client::Serve [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Exiting c++ worker with error: exit status 1
      2021-10-19T03:09:05.825-07:00 [Info] client::Serve [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] After worker exit, stopCalled: false
      2021-10-19T03:09:05.825-07:00 [Info] eventing-consumer [worker_perf-test1_7:/tmp/127.0.0.1:8091_7_1711403943.sock:48984] [lcb,confmon L:183 I:2818644853] Setting initial configuration. Received via CCCP (bucket="eventing-bucket-1", rev=1:139, address="172.23.96.16")
      2021-10-19T03:09:05.825-07:00 [Info] client::Serve [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Informing Eventing.Producer to stop Eventing.Consumer instance: consumer => app: perf-test1 name: worker_perf-test1_16 tcpPort: /tmp/127.0.0.1:8091_16_1711403943.sock ospid: 48944 dcpEventProcessed:  v8EventProcessed: log_level:1 thr_map:1 thr_count:1 v8_init:1 v8_load:1 vb_map:1
      2021-10-19T03:09:05.825-07:00 [Info] Producer::KillAndRespawnEventingConsumer [perf-test1:11] IndexToPurge: 1 ConsumerIndex: 16 Shutting down Eventing.Consumer instance: consumer => app: perf-test1 name: worker_perf-test1_16 tcpPort: /tmp/127.0.0.1:8091_16_1711403943.sock ospid: 48944 dcpEventProcessed:  v8EventProcessed: thr_map:1 thr_count:1 v8_init:1 v8_load:1 vb_map:1 log_level:1
      2021-10-19T03:09:05.825-07:00 [Info] Supervisor::Remove Token id: {8589966985}
      2021-10-19T03:09:05.825-07:00 [Info] Producer::KillAndRespawnEventingConsumer [perf-test1:11] IndexToPurge: 1 ConsumerIndex: 16 Closing down listener handles
      2021-10-19T03:09:05.825-07:00 [Info] DCPT[eventing:FkUq1baH-295655:{eventing:FkUq1baH-295654:worker_perf-test1_21_172.23.96.16:11210_172.23.97.177:8096}/0] ##abcd feed started ... collectionAware: true
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Gracefully shutting down consumer routine
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::RemoveSupervisorToken [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Removing supervisor token
      2021-10-19T03:09:05.825-07:00 [Info] Supervisor::Remove Token id: {139139760521216}
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Requested to remove supervision of eventing-consumer
      2021-10-19T03:09:05.825-07:00 [Error] Producer::handleV8Consumer [perf-test1:11] Accept failed in main loop, err: accept unix /tmp/127.0.0.1:8091_16_1711403943.sock: use of closed network connection
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Stopped checkpoint, restart vb dcp stream and stats tickers
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::sendMessageLoop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Exiting send message routine
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Sent signal over channel to stop timer routines
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Sent signal to stop cpp worker stat collection routine
      2021-10-19T03:09:05.825-07:00 [Info] Consumer::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Sent signal over channel to stop checkpointing routine
      2021-10-19T03:09:05.825-07:00 [Error] Producer::handleV8Consumer [perf-test1:11] Accept failed in feedback loop, err: accept unix /tmp/f_127.0.0.1:8091_16_1711403943.sock: use of closed network connection
      2021-10-19T03:09:05.825-07:00 [Info] Producer::KillAndRespawnEventingConsumer [perf-test1:11] ConsumerIndex: 16 respawning the Eventing.Consumer instance
      2021-10-19T03:09:05.825-07:00 [Info] DCPT[eventing:FkUq1baH-295570:{eventing:FkUq1baH-295569:worker_perf-test1_16_172.23.96.16:11210_172.23.97.177:8096}/0] doReceive(): connection closed
      2021-10-19T03:09:05.825-07:00 [Info] client::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Exiting c++ worker
      2021-10-19T03:09:05.825-07:00 [Error] client::Stop [worker_perf-test1_16:/tmp/127.0.0.1:8091_16_1711403943.sock:48944] Unable to notify  c++ worker, err: os: process already finished
      2021-10-19T03:09:05.825-07:00 [Info] DCPT[eventing:FkUq1baH-295570:{eventing:FkUq1baH-295569:worker_perf-test1_16_172.23.96.16:11210_172.23.97.177:8096}/0] ##abcd ... stopped 

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              vikas.chaudhary Vikas Chaudhary
              vikas.chaudhary Vikas Chaudhary
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty