Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-9939

ep worker stats unit test sporadically fails on centos (description is truncated)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • 3.0
    • 3.0
    • couchbase-bucket
    • Security Level: Public
    • None
    • Untriaged
    • Centos 64-bit

    Description

      It looks like this test is failing sometimes because the task description is getting truncated. See some of the debugging I printed out below.

      Also we need to add the following task to the list:

      tasklist.insert("Tap connection notifier");

      nonio_worker_7:log:0:runtime : 22
      nonio_worker_7:log:0:starttime : 0
      nonio_worker_7:log:0:task : Running a flusher loop: shard 0
      nonio_worker_7:log:10:runtime : 17
      nonio_worker_7:log:10:starttime : 0
      nonio_worker_7:log:10:task : Running a flusher loop: shard 0
      nonio_worker_7:log:11:runtime : 14
      nonio_worker_7:log:11:starttime : 0
      nonio_worker_7:log:11:task : Running a flusher loop: shard 0
      nonio_worker_7:log:12:runtime : 46
      nonio_worker_7:log:12:starttime : 0
      nonio_worker_7:log:12:task : Running a flusher loop: shard 0
      nonio_worker_7:log:13:runtime : 57
      nonio_worker_7:log:13:starttime : 0
      nonio_worker_7:log:13:task : Running a flusher loop: shard 0
      nonio_worker_7:log:14:runtime : 10
      nonio_worker_7:log:14:starttime : 0
      nonio_worker_7:log:14:task : Running a flusher loop: shard 0
      nonio_worker_7:log:15:runtime : 14
      nonio_worker_7:log:15:starttime : 0
      nonio_worker_7:log:15:task : Running a flusher loop: shard 0
      nonio_worker_7:log:16:runtime : 13
      nonio_worker_7:log:16:starttime : 0
      nonio_worker_7:log:16:task : Running a flusher loop: shard 0
      nonio_worker_7:log:17:runtime : 13
      nonio_worker_7:log:17:starttime : 0
      nonio_worker_7:log:17:task : Running a flusher loop: shard 0
      nonio_worker_7:log:18:runtime : 26
      nonio_worker_7:log:18:starttime : 0
      nonio_worker_7:log:18:task : Running a flusher loop: shard 0
      nonio_worker_7:log:19:runtime : 12
      nonio_worker_7:log:19:starttime : 0
      nonio_worker_7:log:19:task : Running a flusher loop: shard 0
      nonio_worker_7:log:1:runtime : 8
      nonio_worker_7:log:1:starttime : 0
      nonio_worker_7:log:1:task : Running a flusher loop: shard 0
      nonio_worker_7:log:2:runtime : 33
      nonio_worker_7:log:2:starttime : 0
      nonio_worker_7:log:2:task : Running a flusher loop: shard 0
      nonio_worker_7:log:3:runtime : 31
      nonio_worker_7:log:3:starttime : 0
      nonio_worker_7:log:3:task : Running a flusher loop: shard 0
      nonio_worker_7:log:4:runtime : 54
      nonio_worker_7:log:4:starttime : 0
      nonio_worker_7:log:4:task : Running a flusher loop: shard 0
      nonio_worker_7:log:5:runtime : 14
      nonio_worker_7:log:5:starttime : 0
      nonio_worker_7:log:5:task : Running a flusher loop: shard 0
      nonio_worker_7:log:6:runtime : 9
      nonio_worker_7:log:6:starttime : 0
      nonio_worker_7:log:6:task : Running a flusher loop: shard 0
      nonio_worker_7:log:7:runtime : 51
      nonio_worker_7:log:7:starttime : 0
      nonio_worker_7:log:7:task : Running a flusher loop: shard 0
      nonio_worker_7:log:8:runtime : 19
      nonio_worker_7:log:8:starttime : 0
      nonio_worker_7:log:8:task : Running a flusher loop: shard 0
      nonio_worker_7:log:9:runtime : 10
      nonio_worker_7:log:9:starttime : 0
      nonio_worker_7:log:9:task : Running a flusher loop: shard 0
      nonio_worker_7:runtime : 16172
      nonio_worker_7:state : running
      nonio_worker_7:task : Not currently running any task
      reader_worker_0:log:0:runtime : 7
      reader_worker_0:log:0:starttime : 0
      reader_worker_0:log:0:task : Batching background fetch
      reader_worker_0:log:1:runtime : 4
      reader_worker_0:log:1:starttime : 0
      reader_worker_0:log:1:task : Batching background fetch
      reader_worker_0:log:2:runtime : 532148
      reader_worker_0:log:2:starttime : 0
      reader_worker_0:log:2:task : Warmup - initialize
      reader_worker_0:log:3:runtime : 14
      reader_worker_0:log:3:starttime : 0
      reader_worker_0:log:3:task : Warmup - estimate database item count
      reader_worker_0:log:4:runtime : 93
      reader_worker_0:log:4:starttime : 0
      reader_worker_0:log:4:task : Wa
      reader_worker_0:log:5:runtime : 18
      reader_worker_0:log:5:starttime : 0
      reader_worker_0:log:5:task : Running a flusher loop: shard 0
      reader_worker_0:log:6:runtime : 14
      reader_worker_0:log:6:starttime : 0
      reader_worker_0:log:6:task :
      reader_worker_0:log:7:runtime : 5
      reader_worker_0:log:7:starttime : 0
      reader_worker_0:log:7:task : Wa
      reader_worker_0:log:8:runtime : 4
      reader_worker_0:log:8:starttime : 0
      reader_worker_0:log:8:task : W
      reader_worker_0:runtime : 3464
      reader_worker_0:slow:0:runtime : 532148
      reader_worker_0:slow:0:starttime : 0
      reader_worker_0:slow:0:task : Warmup - initialize
      reader_worker_0:state : running
      reader_worker_0:task : War
      reader_worker_1:log:0:runtime : 21
      reader_worker_1:log:0:starttime : 0
      reader_worker_1:log:0:task : Running a flusher loop: shard 0
      reader_worker_1:log:10:runtime : 98
      reader_worker_1:log:10:starttime : 0
      reader_worker_1:log:10:task : Running a flusher loop: shard 0
      reader_worker_1:log:11:runtime : 17
      reader_worker_1:log:11:starttime : 0
      reader_worker_1:log:11:task : Running a flusher loop: shard 0
      reader_worker_1:log:12:runtime : 20
      reader_worker_1:log:12:starttime : 0
      reader_worker_1:log:12:task : Running a flusher loop: shard 0
      reader_worker_1:log:13:runtime : 53
      reader_worker_1:log:13:starttime : 0
      reader_worker_1:log:13:task : Running a flusher loop: shard 0
      reader_worker_1:log:14:runtime : 83
      reader_worker_1:log:14:starttime : 0
      reader_worker_1:log:14:task : Running a flusher loop: shard 0
      reader_worker_1:log:15:runtime : 25
      reader_worker_1:log:15:starttime : 0
      reader_worker_1:log:15:task : Running a flusher loop: shard 0
      reader_worker_1:log:16:runtime : 20
      reader_worker_1:log:16:starttime : 0
      reader_worker_1:log:16:task : Running a flusher loop: shard 0
      reader_worker_1:log:17:runtime : 175
      reader_worker_1:log:17:starttime : 0
      reader_worker_1:log:17:task : Warmup - estimate database item count
      reader_worker_1:log:18:runtime : 79
      reader_worker_1:log:18:starttime : 0
      reader_worker_1:log:18:task : Running a flusher loop: shard 0
      reader_worker_1:log:19:runtime : 17
      reader_worker_1:log:19:starttime : 0
      reader_worker_1:log:19:task : Running a flusher loop: shard 0
      reader_worker_1:log:1:runtime : 27
      reader_worker_1:log:1:starttime : 0
      reader_worker_1:log:1:task : Running a flusher loop: shard 0
      reader_worker_1:log:2:runtime : 12
      reader_worker_1:log:2:starttime : 0
      reader_worker_1:log:2:task : Running a flusher loop: shard 0
      reader_worker_1:log:3:runtime : 28
      reader_worker_1:log:3:starttime : 0
      reader_worker_1:log:3:task : Running a flusher loop: shard 0
      reader_worker_1:log:4:runtime : 21
      reader_worker_1:log:4:starttime : 0
      reader_worker_1:log:4:task : Running a flusher loop: shard 0
      reader_worker_1:log:5:runtime : 34
      reader_worker_1:log:5:starttime : 0
      reader_worker_1:log:5:task : Running a flusher loop: shard 0
      reader_worker_1:log:6:runtime : 17
      reader_worker_1:log:6:starttime : 0
      reader_worker_1:log:6:task : Running a flusher loop: shard 0
      reader_worker_1:log:7:runtime : 32
      reader_worker_1:log:7:starttime : 0
      reader_worker_1:log:7:task : Running a flusher loop: shard 0
      reader_worker_1:log:8:runtime : 44
      reader_worker_1:log:8:starttime : 0
      reader_worker_1:log:8:task : Running a flusher loop: shard 0
      reader_worker_1:log:9:runtime : 21
      reader_worker_1:log:9:starttime : 0
      reader_worker_1:log:9:task : Running a flusher loop: shard 0
      reader_worker_1:runtime : 368
      reader_worker_1:state : running
      reader_worker_1:task : Snapshotting vbucket states for the shard: 2
      reader_worker_2:log:0:runtime : 47
      reader_worker_2:log:0:starttime : 0
      reader_worker_2:log:0:task : Running a flusher loop: shard 0
      reader_worker_2:log:10:runtime : 17
      reader_worker_2:log:10:starttime : 0
      reader_worker_2:log:10:task : Running a flusher loop: shard 0
      reader_worker_2:log:11:runtime : 18
      reader_worker_2:log:11:starttime : 0
      reader_worker_2:log:11:task : Running a flusher loop: shard 0
      reader_worker_2:log:12:runtime : 38
      reader_worker_2:log:12:starttime : 0
      reader_worker_2:log:12:task : Running a flusher loop: shard 0
      reader_worker_2:log:13:runtime : 26
      reader_worker_2:log:13:starttime : 0
      reader_worker_2:log:13:task : Running a flusher loop: shard 0
      reader_worker_2:log:14:runtime : 23
      reader_worker_2:log:14:starttime : 0
      reader_worker_2:log:14:task : Running a flusher loop: shard 0
      reader_worker_2:log:15:runtime : 16
      reader_worker_2:log:15:starttime : 0
      reader_worker_2:log:15:task : Running a flusher loop: shard 0
      reader_worker_2:log:16:runtime : 12
      reader_worker_2:log:16:starttime : 0
      reader_worker_2:log:16:task : Running a flusher loop: shard 0
      reader_worker_2:log:17:runtime : 19
      reader_worker_2:log:17:starttime : 0
      reader_worker_2:log:17:task : Running a flusher loop: shard 0
      reader_worker_2:log:18:runtime : 39
      reader_worker_2:log:18:starttime : 0
      reader_worker_2:log:18:task : Running a flusher loop: shard 0
      reader_worker_2:log:19:runtime : 10
      reader_worker_2:log:19:starttime : 0
      reader_worker_2:log:19:task : W
      reader_worker_2:log:1:runtime : 19
      reader_worker_2:log:1:starttime : 0
      reader_worker_2:log:1:task : Running a flusher loop: shard 0
      reader_worker_2:log:2:runtime : 94
      reader_worker_2:log:2:starttime : 0
      reader_worker_2:log:2:task : Running a flusher loop: shard 0
      reader_worker_2:log:3:runtime : 14
      reader_worker_2:log:3:starttime : 0
      reader_worker_2:log:3:task : Running a flusher loop: shard 0
      reader_worker_2:log:4:runtime : 26
      reader_worker_2:log:4:starttime : 0
      reader_worker_2:log:4:task : Running a flusher loop: shard 0
      reader_worker_2:log:5:runtime : 28
      reader_worker_2:log:5:starttime : 0
      reader_worker_2:log:5:task : Running a flusher loop: shard 0
      reader_worker_2:log:6:runtime : 9
      reader_worker_2:log:6:starttime : 0
      reader_worker_2:log:6:task : Running a flusher loop: shard 0
      reader_worker_2:log:7:runtime : 19
      reader_worker_2:log:7:starttime : 0
      reader_worker_2:log:7:task : Running a flusher loop: shard 0
      reader_worker_2:log:8:runtime : 24
      reader_worker_2:log:8:starttime : 0
      reader_worker_2:log:8:task : Running a flusher loop: shard 0
      reader_worker_2:log:9:runtime : 12
      reader_worker_2:log:9:starttime : 0
      reader_worker_2:log:9:task : Running a flusher loop: shard 0
      reader_worker_2:runtime : 3287
      reader_worker_2:state : running
      reader_worker_2:task : Tap connection notifier
      writer_worker_3:log:0:runtime : 1486
      writer_worker_3:log:0:starttime : 0
      writer_worker_3:log:0:task : Running a flusher loop: shard 0
      writer_worker_3:log:10:runtime : 38
      writer_worker_3:log:10:starttime : 0
      writer_worker_3:log:10:task : Running a flusher loop: shard 0
      writer_worker_3:log:11:runtime : 7
      writer_worker_3:log:11:starttime : 0
      writer_worker_3:log:11:task : Running a flusher loop: shard 0
      writer_worker_3:log:12:runtime : 6
      writer_worker_3:log:12:starttime : 0
      writer_worker_3:log:12:task : Running a flusher loop: shard 0
      writer_worker_3:log:13:runtime : 7
      writer_worker_3:log:13:starttime : 0
      writer_worker_3:log:13:task : Running a flusher loop: shard 0
      writer_worker_3:log:14:runtime : 8
      writer_worker_3:log:14:starttime : 0
      writer_worker_3:log:14:task : Running a flusher loop: shard 0
      writer_worker_3:log:15:runtime : 12
      writer_worker_3:log:15:starttime : 0
      writer_worker_3:log:15:task : Running a flusher loop: shard 0
      writer_worker_3:log:16:runtime : 9
      writer_worker_3:log:16:starttime : 0
      writer_worker_3:log:16:task : Running a flusher loop: shard 0
      writer_worker_3:log:17:runtime : 17
      writer_worker_3:log:17:starttime : 0
      writer_worker_3:log:17:task : Running a flusher loop: shard 0
      writer_worker_3:log:18:runtime : 108
      writer_worker_3:log:18:starttime : 0
      writer_worker_3:log:18:task : Running a flusher loop: shard 0
      writer_worker_3:log:19:runtime : 67
      writer_worker_3:log:19:starttime : 0
      writer_worker_3:log:19:task : Running a flusher loop: shard 0
      writer_worker_3:log:1:runtime : 11
      writer_worker_3:log:1:starttime : 0
      writer_worker_3:log:1:task : Running a flusher loop: shard 0
      writer_worker_3:log:2:runtime : 7
      writer_worker_3:log:2:starttime : 0
      writer_worker_3:log:2:task : Running a flusher loop: shard 0
      writer_worker_3:log:3:runtime : 6
      writer_worker_3:log:3:starttime : 0
      writer_worker_3:log:3:task : Running a flusher loop: shard 0
      writer_worker_3:log:4:runtime : 7
      writer_worker_3:log:4:starttime : 0
      writer_worker_3:log:4:task : Running a flusher loop: shard 0
      writer_worker_3:log:5:runtime : 7
      writer_worker_3:log:5:starttime : 0
      writer_worker_3:log:5:task : Running a flusher loop: shard 0
      writer_worker_3:log:6:runtime : 11
      writer_worker_3:log:6:starttime : 0
      writer_worker_3:log:6:task : Running a flusher loop: shard 0
      writer_worker_3:log:7:runtime : 8
      writer_worker_3:log:7:starttime : 0
      writer_worker_3:log:7:task : Running a flusher loop: shard 0
      writer_worker_3:log:8:runtime : 7
      writer_worker_3:log:8:starttime : 0
      writer_worker_3:log:8:task : Running a flusher loop: shard 0
      writer_worker_3:log:9:runtime : 7
      writer_worker_3:log:9:starttime : 0
      writer_worker_3:log:9:task : Running a flusher loop: shard 0
      writer_worker_3:runtime : 2097
      writer_worker_3:state : running
      writer_worker_3:task : Snapshotting vbucket states for the shard: 0
      writer_worker_4:log:0:runtime : 20
      writer_worker_4:log:0:starttime : 0
      writer_worker_4:log:0:task : Running a flusher loop: shard 0
      writer_worker_4:log:10:runtime : 27
      writer_worker_4:log:10:starttime : 0
      writer_worker_4:log:10:task : Running a flusher loop: shard 0
      writer_worker_4:log:11:runtime : 26
      writer_worker_4:log:11:starttime : 0
      writer_worker_4:log:11:task : Running a flusher loop: shard 0
      writer_worker_4:log:12:runtime : 38
      writer_worker_4:log:12:starttime : 0
      writer_worker_4:log:12:task : Running a flusher loop: shard 0
      writer_worker_4:log:13:runtime : 9
      writer_worker_4:log:13:starttime : 0
      writer_worker_4:log:13:task : Running a flusher loop: shard 0
      writer_worker_4:log:14:runtime : 22
      writer_worker_4:log:14:starttime : 0
      writer_worker_4:log:14:task : Running a flusher loop: shard 0
      writer_worker_4:log:15:runtime : 13
      writer_worker_4:log:15:starttime : 0
      writer_worker_4:log:15:task : Running a flusher loop: shard 0
      writer_worker_4:log:16:runtime : 19
      writer_worker_4:log:16:starttime : 0
      writer_worker_4:log:16:task : Running a flusher loop: shard 0
      writer_worker_4:log:17:runtime : 24
      writer_worker_4:log:17:starttime : 0
      writer_worker_4:log:17:task : Running a flusher loop: shard 0
      writer_worker_4:log:18:runtime : 194
      writer_worker_4:log:18:starttime : 0
      writer_worker_4:log:18:task : Running a flusher loop: shard 0
      writer_worker_4:log:19:runtime : 870
      writer_worker_4:log:19:starttime : 0
      writer_worker_4:log:19:task : Running a flusher loop: shard 0
      writer_worker_4:log:1:runtime : 15
      writer_worker_4:log:1:starttime : 0
      writer_worker_4:log:1:task : Running a flusher loop: shard 0
      writer_worker_4:log:2:runtime : 11
      writer_worker_4:log:2:starttime : 0
      writer_worker_4:log:2:task : Running a flusher loop: shard 0
      writer_worker_4:log:3:runtime : 66
      writer_worker_4:log:3:starttime : 0
      writer_worker_4:log:3:task : Running a flusher loop: shard 0
      writer_worker_4:log:4:runtime : 37
      writer_worker_4:log:4:starttime : 0
      writer_worker_4:log:4:task : Running a flusher loop: shard 0
      writer_worker_4:log:5:runtime : 17
      writer_worker_4:log:5:starttime : 0
      writer_worker_4:log:5:task : Running a flusher loop: shard 0
      writer_worker_4:log:6:runtime : 17
      writer_worker_4:log:6:starttime : 0
      writer_worker_4:log:6:task : Running a flusher loop: shard 0
      writer_worker_4:log:7:runtime : 7
      writer_worker_4:log:7:starttime : 0
      writer_worker_4:log:7:task : Running a flusher loop: shard 0
      writer_worker_4:log:8:runtime : 33
      writer_worker_4:log:8:starttime : 0
      writer_worker_4:log:8:task : Running a flusher loop: shard 0
      writer_worker_4:log:9:runtime : 17
      writer_worker_4:log:9:starttime : 0
      writer_worker_4:log:9:task : Running a flusher loop: shard 0
      writer_worker_4:runtime : 1492
      writer_worker_4:state : running
      writer_worker_4:task : Snapshotting vbucket states for the shard: 3
      writer_worker_5:log:0:runtime : 11
      writer_worker_5:log:0:starttime : 0
      writer_worker_5:log:0:task : Running a flusher loop: shard 0
      writer_worker_5:log:10:runtime : 11
      writer_worker_5:log:10:starttime : 0
      writer_worker_5:log:10:task : Running a flusher loop: shard 0
      writer_worker_5:log:11:runtime : 17
      writer_worker_5:log:11:starttime : 0
      writer_worker_5:log:11:task : Running a flusher loop: shard 0
      writer_worker_5:log:12:runtime : 19
      writer_worker_5:log:12:starttime : 0
      writer_worker_5:log:12:task : Running a flusher loop: shard 0
      writer_worker_5:log:13:runtime : 57
      writer_worker_5:log:13:starttime : 0
      writer_worker_5:log:13:task : Running a flusher loop: shard 0
      writer_worker_5:log:14:runtime : 13
      writer_worker_5:log:14:starttime : 0
      writer_worker_5:log:14:task : Running a flusher loop: shard 0
      writer_worker_5:log:15:runtime : 18
      writer_worker_5:log:15:starttime : 0
      writer_worker_5:log:15:task : Warmup - estimate database item count
      writer_worker_5:log:16:runtime : 1519
      writer_worker_5:log:16:starttime : 0
      writer_worker_5:log:16:task : Warmup - check for access log
      writer_worker_5:log:17:runtime : 283
      writer_worker_5:log:17:starttime : 0
      writer_worker_5:log:17:task : Running a flusher loop: shard 0
      writer_worker_5:log:18:runtime : 19
      writer_worker_5:log:18:starttime : 0
      writer_worker_5:log:18:task : Running a flusher loop: shard 0
      writer_worker_5:log:19:runtime : 102
      writer_worker_5:log:19:starttime : 0
      writer_worker_5:log:19:task : Running a flusher loop: shard 0
      writer_worker_5:log:1:runtime : 28
      writer_worker_5:log:1:starttime : 0
      writer_worker_5:log:1:task : Running a flusher loop: shard 0
      writer_worker_5:log:2:runtime : 9
      writer_worker_5:log:2:starttime : 0
      writer_worker_5:log:2:task : Running a flusher loop: shard 0
      writer_worker_5:log:3:runtime : 7
      writer_worker_5:log:3:starttime : 0
      writer_worker_5:log:3:task : Running a flusher loop: shard 0
      writer_worker_5:log:4:runtime : 25
      writer_worker_5:log:4:starttime : 0
      writer_worker_5:log:4:task : Running a flusher loop: shard 0
      writer_worker_5:log:5:runtime : 36
      writer_worker_5:log:5:starttime : 0
      writer_worker_5:log:5:task : Running a flusher loop: shard 0
      writer_worker_5:log:6:runtime : 29
      writer_worker_5:log:6:starttime : 0
      writer_worker_5:log:6:task : Running a flusher loop: shard 0
      writer_worker_5:log:7:runtime : 17
      writer_worker_5:log:7:starttime : 0
      writer_worker_5:log:7:task : Running a flusher loop: shard 0
      writer_worker_5:log:8:runtime : 57
      writer_worker_5:log:8:starttime : 0
      writer_worker_5:log:8:task : Running a flusher loop: shard 0
      writer_worker_5:log:9:runtime : 27
      writer_worker_5:log:9:starttime : 0
      writer_worker_5:log:9:task : Running a flusher loop: shard 0
      writer_worker_5:runtime : 2005
      writer_worker_5:state : running
      writer_worker_5:task : Updating stat snapshot on disk
      Task 0: 'War'
      /home/jenkins/couchbase/cmake/ep-engine/tests/ep_testsuite.cc:4919 Test failed: `worker_0's Current task incorrect' (tasklist.find(worker_0_task)!=tasklist.end())
      DIED

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            venu Venu Uppalapati (Inactive)
            mikew Mike Wiederhold [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty