Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50804

[System Test] - Pausing handler gets stuck after enforcing TLS

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • No

    Description

      Script to Repro
      1. Run CC longevity on 7.0.3 for 3-4 days

      ./sequoia -client 172.23.104.254:2375 -provider file:centos_second_cluster.yml -test tests/integration/cheshirecat/test_cheshirecat_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas_scale3.yml -scope tests/integration/cheshirecat/scope_cheshirecat_with_backup.yml -scale 3 -repeat 0 -log_level 0 -version 7.0.3-7031 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=604800 -show_topology=true

      2. Change the encryption level to strict.
      3. Pause all the handlers. One of handlers got stuck in pausing state.

      Errors from one of the node 172.23.96.48

      2022-02-03T23:20:49.697-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:2818181914] <172.23.123.31:11210> (SOCK=c60f62e9b1a0511d) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.697-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:2818181914] <172.23.120.74:11210> (SOCK=8c19fee7be34d54e) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.697-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:2818181914] <172.23.120.74:11210> (SOCK=8c19fee7be34d54e) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.697-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:2818181914] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff81513a0,IX=1) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.697-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:2818181914] <172.23.123.31:11210> (SOCK=c60f62e9b1a0511d) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.697-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:2818181914] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff815fc00,IX=6) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.706-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:1878817049] <172.23.96.243:11210> (SOCK=2a2600f9b8736da5) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.706-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:1878817049] <172.23.96.243:11210> (SOCK=2a2600f9b8736da5) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.706-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:1878817049] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff80edec0,IX=8) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.726-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:1878817049] <172.23.123.31:11210> (SOCK=4e69a37f86b82300) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.726-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:1878817049] <172.23.123.31:11210> (SOCK=4e69a37f86b82300) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.726-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:1878817049] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff80e8120,IX=6) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.737-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:1878817049] <172.23.123.31:11210> (SOCK=9c587ee8da54ad7b) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.737-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:1878817049] <172.23.123.31:11210> (SOCK=9c587ee8da54ad7b) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.737-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:1878817049] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff80e8120,IX=6) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.748-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:1878817049] <172.23.123.31:11210> (SOCK=af5dba5bb7e7e8aa) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.748-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:1878817049] <172.23.123.31:11210> (SOCK=af5dba5bb7e7e8aa) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.748-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:1878817049] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff80e8120,IX=6) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.755-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:2818181914] <172.23.123.31:11210> (SOCK=586bd18762ca1974) Starting. Timeout=5000000us
      		 
      2022-02-03T23:20:49.755-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:162 I:2818181914] <172.23.123.31:11210> (SOCK=586bd18762ca1974) Failed to establish connection: LCB_ERR_CONNECTION_REFUSED (1024), os errno=111
      		 
      2022-02-03T23:20:49.755-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,server L:965 I:2818181914] <NOHOST:NOPORT> (CTX=(nil),,SRV=0x7f0ff815fc00,IX=6) Connection attempt failed. Received LCB_ERR_CONNECTION_REFUSED (1024) from libcouchbase, received 0 (Success) from operating system
      		 
      2022-02-03T23:20:49.767-08:00 [Info] eventing-consumer [worker_timers_0_0:/tmp/127.0.0.1:8091_0_4019305715.sock:32407] [lcb,connection L:485 I:2818181914] <172.23.123.31:11210> (SOCK=7dce03f655841e63) Starting. Timeout=5000000us

      cbcollect_info attached.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-50456 Eventing: Timer handler stuck in pausing state after enforcing TLS

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Balakumaran.Gopal Balakumaran Gopal
              Balakumaran.Gopal Balakumaran Gopal
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty