Details
-
Bug
-
Resolution: Fixed
-
Major
-
5.0.0
-
Untriaged
-
Centos 64-bit
-
-
Yes
-
KV Spock Beta
Description
Multiple tests indicate a significant drop in write heavy performance with high rate mutations.
For instance, take a look at pillowfight benchmarks
http://showfast.sc.couchbase.com/#/timeline/Linux/kv/max_ops
The latest known good build is 1544, builds 1545 - 1549 were broken due to MB-22131, and the bug eventually showed up in the build 1550.
Although pillowfight is probably the easiest way to reproduce this issue, our test for the initial data load produces the most odd results.
The test inserts 500M brand new items into 4 Couchbase Server nodes. The following stats look unexpected:
cmd_set: 124999717
|
cmd_total_ops: 125000000
|
cmd_total_sets: 125000000
|
|
|
ep_bg_wait: 43530755849
|
|
bg_wait (91175924 total)
|
1us - 2us : ( 0.00%) 796
|
2us - 4us : ( 0.09%) 82868
|
4us - 8us : ( 0.68%) 537341
|
8us - 16us : ( 12.02%) 10340101 ####
|
16us - 32us : ( 32.19%) 18392876 #######
|
32us - 64us : ( 43.45%) 10258948 ####
|
64us - 128us : ( 55.50%) 10986997 ####
|
128us - 256us : ( 66.91%) 10404870 ####
|
256us - 512us : ( 79.10%) 11113654 ####
|
512us - 1ms : ( 90.21%) 10134489 ####
|
1ms - 2ms : ( 97.34%) 6498980 ##
|
2ms - 4ms : ( 99.57%) 2032286
|
4ms - 8ms : ( 99.84%) 244775
|
8ms - 16ms : ( 99.89%) 45752
|
16ms - 32ms : ( 99.92%) 26851
|
32ms - 65ms : ( 99.95%) 28525
|
65ms - 131ms : ( 99.97%) 14988
|
131ms - 262ms : ( 99.98%) 14164
|
262ms - 524ms : (100.00%) 16123
|
524ms - 1s : (100.00%) 402
|
1s - 2s : (100.00%) 138
|
Avg : ( 348us)
|
This report illustrates how bg fetches impact front-end ops:
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=hera_470-1550_load_00b0
pillowfight steps:
cbc-pillowfight --spec couchbase://172.23.133.11/bucket-1 --password password --batch-size 1000 --num-items 20000000 --num-threads 50 --min-size 256 --max-size 256 --populate-only
|
|
|
cbc-pillowfight --spec couchbase://172.23.133.11/bucket-1 --password password --batch-size 1000 --num-items 20000000 --num-threads 50 --min-size 256 --max-size 256 --set-pct 50 --num-cycles 40000 --no-population
|
Attachments
Issue Links
- blocks
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-