Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48623

on_update failing with LCB_ERR_TIMEOUT

    XMLWordPrintable

Details

    Description

      Build: 7.1.0-1345 not seen on 7.1.0-1250

      Test: 
      Timers executed/sec, 1 bucket x 20M x 1KB, 4KV + 1Eventing node, 5 timers-NOOP

      Run: 

      http://perf.jenkins.couchbase.com/job/themis/12770/console

      Seeing the following error in the logs for multiple tests 

      2021-09-26T08:34:07.131-07:00 [Info] eventing-consumer [worker_perf-test5_0:/tmp/127.0.0.1:8091_0_3084538691.sock:43000] [lcb,server L:818 I:2407863522] Failing command with error LCB_ERR_TIMEOUT (201): {"b":"eventing","i":"92dde93f8f8514e2/ef48a6df59803a06/4deb7","l":"172.23.97.177:47436","r":"172.23.96.20:11210","s":"kv:set","t":5000000}
      2021-09-26T08:34:07.131-07:00 [Info] eventing-consumer [worker_perf-test1_2:/tmp/127.0.0.1:8091_2_2860022324.sock:42518] [lcb,server L:818 I:2507039759] Failing command with error LCB_ERR_TIMEOUT (201): {"b":"eventing","i":"90bfc60a956e640f/ff1bc8b93a32b07f/4f291","l":"172.23.97.177:46722","r":"172.23.96.20:11210","s":"kv:set","t":5000000}
      2021-09-26T08:34:07.185-07:00 [Info] eventing-consumer [worker_perf-test1_4:/tmp/127.0.0.1:8091_4_2860022324.sock:42543] [lcb,server L:818 I:3643205127] Failing command with error LCB_ERR_TIMEOUT (201): {"b":"eventing","i":"0e2fa779d926e607/f81453f5d0985093/532e0","l":"172.23.97.177:46720","r":"172.23.96.20:11210","s":"kv:incr","t":5000000}
      2021-09-26T08:34:07.186-07:00 [Info] eventing-consumer [worker_perf-test2_3:/tmp/127.0.0.1:8091_3_3574107999.sock:42647] [lcb,server L:818 I:2022674160] Failing command with error LCB_ERR_TIMEOUT (201): {"b":"eventing","i":"42cc29bb788f8ef0/24801a71ed2c8d57/4c0db","l":"172.23.97.177:46926","r":"172.23.96.20:11210","s":"kv:set","t":5000000}
      2021-09-26T08:34:07.228-07:00 [Info] eventing-consumer [worker_perf-test1_4:/tmp/127.0.0.1:8091_4_2860022324.sock:42543] Timer Creation failed with message: Set Timer Failed with KV Error
      2021-09-26T08:34:07.228-07:00 [Info] eventing-consumer [worker_perf-test1_4:/tmp/127.0.0.1:8091_4_2860022324.sock:42543] file : /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/eventing/v8_consumer/src/v8worker.cc line : 901 caller : SendUpdate : Returning empty valueError executing on_doc_update 
      2021-09-26T08:34:07.228-07:00 [Info] eventing-consumer [worker_perf-test1_4:/tmp/127.0.0.1:8091_4_2860022324.sock:42543] {  "message": {    "code": 5,    "desc": "LCB_ERR_TIMEOUT (201): The request was not completed by the user-defined timeout",    "name": "LCB_ERR_TIMEOUT"  },  "stack": "Error\n    at OnUpdate (perf-test1.js:5:5)"} Line: 4 Code: createTimer(timerCallback, fireAt, meta.id); Stack: Error    at OnUpdate (perf-test1.js:5:5) 
      2021-09-26T08:34:07.362-07:00 [Info] eventing-consumer [worker_perf-test3_1:/tmp/127.0.0.1:8091_1_1490886059.sock:42754] [lcb,cccp L:187 I:832627505] <NOHOST:NOPORT> (CTX=(nil),) Could not get configuration: LCB_ERR_TIMEOUT (201)
      2021-09-26T08:34:07.370-07:00 [Info] eventing-consumer [worker_perf-test1_1:/tmp/127.0.0.1:8091_1_2860022324.sock:42501] [lcb,cccp L:187 I:2675563147] <NOHOST:NOPORT> (CTX=(nil),) Could not get configuration: LCB_ERR_TIMEOUT (201)
      2021-09-26T08:34:07.443-07:00 [Info] DCPT[eventing:9tcPjgtG-47:{eventing:9tcPjgtG-46:worker_perf-test1_4_172.23.96.23:11210_172.23.97.177:8096}/0] DCP-socket -> eventing blocked 2.996807549s (0.529047%)
      2021-09-26T08:34:07.483-07:00 [Info] eventing-consumer [worker_perf-test3_3:/tmp/127.0.0.1:8091_3_1490886059.sock:42773] [lcb,cccp L:187 I:1666361031] <NOHOST:NOPORT> (CTX=(nil),) Could not get configuration: LCB_ERR_TIMEOUT (201)
      2021-09-26T08:34:07.483-07:00 [Info] eventing-consumer [worker_perf-test3_3:/tmp/127.0.0.1:8091_3_1490886059.sock:42773] [lcb,confmon L:216 I:1666361031] Provider 'CCCP' failed: LCB_ERR_TIMEOUT (201) 

      Change log : http://changelog.build.couchbase.com/?product=couchbase-server&fromVersion=7.1.0&fromBuild=1250&toVersion=7.1.0&toBuild=1345&f_analytics-dcp-client=off&f_asterixdb=off&f_backup=off&f_cbas=off&f_cbas-core=off&f_cbbs=off&f_cbft=off&f_cbftx=off&f_cbgt=off&f_couchbase-cli=off&f_eventing=on&f_go-couchbase=off&f_goxdcr=off&f_indexing=off&f_kv_engine=off&f_n1fty=off&f_ns_server=off&f_plasma=off&f_product-metadata=off&f_query=off&f_query-ui=off&f_testrunner=off&f_tlm=off&f_voltron=off&f_HdrHistogram_c=off&f_build=off&f_cbas-ui=off&f_cbauth=off&f_cbsummary=off&f_chronicle=off&f_couchdb=off&f_couchstore=off&f_docloader=off&f_eventing-ee=off&f_forestdb=off&f_geocouch=off&f_gomemcached=off&f_gometa=off&f_magma=off&f_nitro=off&f_phosphor=off&f_platform=off&f_query-ee=off&f_subjson=off&f_text=off&f_udf-api=off&f_goutils=off 

      Attachments

        1. cp-memory-7.1.0-1250.png
          cp-memory-7.1.0-1250.png
          49 kB
        2. cp-memory-7.1.0-1345.png
          cp-memory-7.1.0-1345.png
          51 kB
        3. memory-7.1.0-1250.png
          memory-7.1.0-1250.png
          198 kB
        4. memory-7.1.0-1345.png
          memory-7.1.0-1345.png
          216 kB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              vikas.chaudhary Vikas Chaudhary
              vikas.chaudhary Vikas Chaudhary
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty