Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7838

High variation in rebalance time during large scale view perf. tests

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Won't Fix
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.0.1
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Labels:
      None

      Description

      Brief test spec:
      3 -> 4 nodes rebalance in
      60M items (~70% DGM, 1 replica)
      1 ddocs with 3 views, no index replica
      6K ops/sec (80% read), 600 queries/sec

      Build 160 (known to have ep-engine issue):
      10.9h : http://172.23.96.10:8080/job/apollo-views/33/artifact/
      6.8h : http://172.23.96.10:8080/job/apollo-views/38/artifact/

      Build 168:
      8.8h: http://172.23.96.10:8080/job/apollo-views/54/artifact/
      5.7h: http://172.23.96.10:8080/job/apollo-views/55/artifact/

      Probably related issue: MB-7835.

      1. master_events_faster.log.svg
        287 kB
        Aleksey Kondratenko
      2. master_events_slower.log.svg
        309 kB
        Aleksey Kondratenko
      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        xiaoqin Xiaoqin Ma (Inactive) added a comment -
        Show
        xiaoqin Xiaoqin Ma (Inactive) added a comment - There are nothing here: http://172.23.96.10:8080/job/apollo-views/34/artifact/
        Hide
        xiaoqin Xiaoqin Ma (Inactive) added a comment -

        For build 168, we see the "access scanner" run one time at the 8.8h run.
        I need logs from other slower/faster run pairs from the same build to confirm that.

        Show
        xiaoqin Xiaoqin Ma (Inactive) added a comment - For build 168, we see the "access scanner" run one time at the 8.8h run. I need logs from other slower/faster run pairs from the same build to confirm that.
        Hide
        xiaoqin Xiaoqin Ma (Inactive) added a comment -

        I have verified from more logs that when "access scanner" runs half minute or more, we see the overall performance degration.

        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_access_scanner_last_runtime: 51
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_access_scanner_num_items: 49951820
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_num_access_scanner_runs: 1
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_access_scanner_last_runtime: 49
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_access_scanner_num_items: 49952524
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_num_access_scanner_runs: 1
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_access_scanner_last_runtime: 49
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_access_scanner_num_items: 49956514
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_num_access_scanner_runs: 1
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_access_scanner_last_runtime: 52
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_access_scanner_num_items: 49948930
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03
        258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_num_access_scanner_runs: 1
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_access_scanner_last_runtime: 4
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_access_scanner_num_items: 4671992
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_num_access_scanner_runs: 1
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_access_scanner_last_runtime: 5
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_access_scanner_num_items: 4528459
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_num_access_scanner_runs: 1
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_access_scanner_last_runtime: 4
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_access_scanner_num_items: 4223707
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_num_access_scanner_runs: 1
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_access_scanner_last_runtime: 3
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_access_scanner_num_items: 3746895
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00
        261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_num_access_scanner_runs: 1
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_access_scanner_last_runtime: 29
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_access_scanner_num_items: 30002618
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_access_scanner_task_time: 2012-03-21 10:00:00
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_num_access_scanner_runs: 1
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_access_scanner_last_runtime: 28
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_access_scanner_num_items: 25562089
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_access_scanner_task_time: 2013-02-19 10:00:00
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_num_access_scanner_runs: 1
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_access_scanner_last_runtime: 32
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_access_scanner_num_items: 29853982
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_access_scanner_task_time: 2013-02-19 10:00:00
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_num_access_scanner_runs: 1
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_access_scanner_last_runtime: 0
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_access_scanner_num_items: 0
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_access_scanner_task_time: 2013-02-19 10:00:00
        33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_num_access_scanner_runs: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_access_scanner_last_runtime: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_access_scanner_num_items: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_access_scanner_task_time: 2012-03-23 10:00:00
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_num_access_scanner_runs: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_access_scanner_last_runtime: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_access_scanner_num_items: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_access_scanner_task_time: 2013-02-21 10:00:00
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_num_access_scanner_runs: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_access_scanner_last_runtime: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_access_scanner_num_items: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_access_scanner_task_time: 2013-02-21 10:00:00
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_num_access_scanner_runs: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_access_scanner_last_runtime: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_access_scanner_num_items: 0
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_access_scanner_task_time: 2013-02-21 10:00:00
        38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_num_access_scanner_runs: 0
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_access_scanner_last_runtime: 39
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_access_scanner_num_items: 32575724
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_access_scanner_task_time: 2012-03-30 10:00:00
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_num_access_scanner_runs: 1
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_access_scanner_last_runtime: 46
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_access_scanner_num_items: 40083650
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:02
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_num_access_scanner_runs: 1
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_access_scanner_last_runtime: 71
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_access_scanner_num_items: 37211086
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:01
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_num_access_scanner_runs: 1
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_access_scanner_last_runtime: 10
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_access_scanner_num_items: 10329930
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:01
        54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_num_access_scanner_runs: 1
        55_5.7/15/stats.log: ep_access_scanner_last_runtime: 0
        55_5.7/15/stats.log: ep_access_scanner_num_items: 0
        55_5.7/15/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:00
        55_5.7/15/stats.log: ep_num_access_scanner_runs: 0
        55_5.7/16/stats.log: ep_access_scanner_last_runtime: 0
        55_5.7/16/stats.log: ep_access_scanner_num_items: 0
        55_5.7/16/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:00
        55_5.7/16/stats.log: ep_num_access_scanner_runs: 0
        55_5.7/17/stats.log: ep_access_scanner_last_runtime: 0
        55_5.7/17/stats.log: ep_access_scanner_num_items: 0
        55_5.7/17/stats.log: ep_access_scanner_task_time: 2012-03-30 10:00:00
        55_5.7/17/stats.log: ep_num_access_scanner_runs: 0
        55_5.7/18/stats.log: ep_access_scanner_last_runtime: 0
        55_5.7/18/stats.log: ep_access_scanner_num_items: 0
        55_5.7/18/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:00
        55_5.7/18/stats.log: ep_num_access_scanner_runs: 0

        Show
        xiaoqin Xiaoqin Ma (Inactive) added a comment - I have verified from more logs that when "access scanner" runs half minute or more, we see the overall performance degration. 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_access_scanner_last_runtime: 51 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_access_scanner_num_items: 49951820 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124438/stats.log: ep_num_access_scanner_runs: 1 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_access_scanner_last_runtime: 49 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_access_scanner_num_items: 49952524 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-124640/stats.log: ep_num_access_scanner_runs: 1 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_access_scanner_last_runtime: 49 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_access_scanner_num_items: 49956514 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125103/stats.log: ep_num_access_scanner_runs: 1 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_access_scanner_last_runtime: 52 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_access_scanner_num_items: 49948930 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_access_scanner_task_time: 2013-02-25 10:00:03 258_slow/cbcollect_info_ns_1@127.0.0.1_20130224-125106/stats.log: ep_num_access_scanner_runs: 1 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_access_scanner_last_runtime: 4 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_access_scanner_num_items: 4671992 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201330/stats.log: ep_num_access_scanner_runs: 1 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_access_scanner_last_runtime: 5 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_access_scanner_num_items: 4528459 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-201535/stats.log: ep_num_access_scanner_runs: 1 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_access_scanner_last_runtime: 4 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_access_scanner_num_items: 4223707 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202011/stats.log: ep_num_access_scanner_runs: 1 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_access_scanner_last_runtime: 3 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_access_scanner_num_items: 3746895 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_access_scanner_task_time: 2013-02-26 10:00:00 261_fast/cbcollect_info_ns_1@127.0.0.1_20130225-202030/stats.log: ep_num_access_scanner_runs: 1 33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_access_scanner_last_runtime: 29 33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_access_scanner_num_items: 30002618 33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_access_scanner_task_time: 2012-03-21 10:00:00 33_10.8/cbcollect_info_ns_1@127.0.0.1_20120321-001532/stats.log: ep_num_access_scanner_runs: 1 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_access_scanner_last_runtime: 28 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_access_scanner_num_items: 25562089 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_access_scanner_task_time: 2013-02-19 10:00:00 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001227/stats.log: ep_num_access_scanner_runs: 1 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_access_scanner_last_runtime: 32 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_access_scanner_num_items: 29853982 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_access_scanner_task_time: 2013-02-19 10:00:00 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001559/stats.log: ep_num_access_scanner_runs: 1 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_access_scanner_last_runtime: 0 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_access_scanner_num_items: 0 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_access_scanner_task_time: 2013-02-19 10:00:00 33_10.8/cbcollect_info_ns_1@127.0.0.1_20130219-001856/stats.log: ep_num_access_scanner_runs: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_access_scanner_last_runtime: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_access_scanner_num_items: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_access_scanner_task_time: 2012-03-23 10:00:00 38_6.8/cbcollect_info_ns_1@127.0.0.1_20120322-205521/stats.log: ep_num_access_scanner_runs: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_access_scanner_last_runtime: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_access_scanner_num_items: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_access_scanner_task_time: 2013-02-21 10:00:00 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205317/stats.log: ep_num_access_scanner_runs: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_access_scanner_last_runtime: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_access_scanner_num_items: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_access_scanner_task_time: 2013-02-21 10:00:00 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205547/stats.log: ep_num_access_scanner_runs: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_access_scanner_last_runtime: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_access_scanner_num_items: 0 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_access_scanner_task_time: 2013-02-21 10:00:00 38_6.8/cbcollect_info_ns_1@127.0.0.1_20130220-205846/stats.log: ep_num_access_scanner_runs: 0 54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_access_scanner_last_runtime: 39 54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_access_scanner_num_items: 32575724 54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_access_scanner_task_time: 2012-03-30 10:00:00 54_8.8/cbcollect_info_ns_1@127.0.0.1_20120329-182432/stats.log: ep_num_access_scanner_runs: 1 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_access_scanner_last_runtime: 46 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_access_scanner_num_items: 40083650 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:02 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182227/stats.log: ep_num_access_scanner_runs: 1 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_access_scanner_last_runtime: 71 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_access_scanner_num_items: 37211086 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:01 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182452/stats.log: ep_num_access_scanner_runs: 1 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_access_scanner_last_runtime: 10 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_access_scanner_num_items: 10329930 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:01 54_8.8/cbcollect_info_ns_1@127.0.0.1_20130227-182758/stats.log: ep_num_access_scanner_runs: 1 55_5.7/15/stats.log: ep_access_scanner_last_runtime: 0 55_5.7/15/stats.log: ep_access_scanner_num_items: 0 55_5.7/15/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:00 55_5.7/15/stats.log: ep_num_access_scanner_runs: 0 55_5.7/16/stats.log: ep_access_scanner_last_runtime: 0 55_5.7/16/stats.log: ep_access_scanner_num_items: 0 55_5.7/16/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:00 55_5.7/16/stats.log: ep_num_access_scanner_runs: 0 55_5.7/17/stats.log: ep_access_scanner_last_runtime: 0 55_5.7/17/stats.log: ep_access_scanner_num_items: 0 55_5.7/17/stats.log: ep_access_scanner_task_time: 2012-03-30 10:00:00 55_5.7/17/stats.log: ep_num_access_scanner_runs: 0 55_5.7/18/stats.log: ep_access_scanner_last_runtime: 0 55_5.7/18/stats.log: ep_access_scanner_num_items: 0 55_5.7/18/stats.log: ep_access_scanner_task_time: 2013-02-28 10:00:00 55_5.7/18/stats.log: ep_num_access_scanner_runs: 0
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Here's what I have so far.

        We can visibly see how first batch of moves affecting first node is many times slower in slower path. But outside of backfill stage.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Here's what I have so far. We can visibly see how first batch of moves affecting first node is many times slower in slower path. But outside of backfill stage.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Hm. I was silently assuming ep-engine folks (i.e. Xiaoqin Ma) found something here. Do you still need me here ?

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Hm. I was silently assuming ep-engine folks (i.e. Xiaoqin Ma) found something here. Do you still need me here ?
        Hide
        pavelpaulau Pavel Paulau added a comment -

        May I ask ep-engine team to share their resolution?

        Show
        pavelpaulau Pavel Paulau added a comment - May I ask ep-engine team to share their resolution?
        Hide
        mikew Mike Wiederhold added a comment -

        This is a result of the background fetcher change that was initially put into 2.0.1. This change was reverted.

        Show
        mikew Mike Wiederhold added a comment - This is a result of the background fetcher change that was initially put into 2.0.1. This change was reverted.

          People

          • Assignee:
            mikew Mike Wiederhold
            Reporter:
            pavelpaulau Pavel Paulau
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes