Details
-
Bug
-
Resolution: Fixed
-
Major
-
master
-
None
-
Untriaged
-
Unknown
Description
From logs of Perf Test on Hera-Pl http://perf.jenkins.couchbase.com/job/hera-pl/25/, long list of Writes Stalling spanning the entire Random Access Phase.
Paolos-MBP:cbcollect_info_ns_1@172.23.96.117_20171217-013552 paoloc$ grep "Stalling" kv_rocks.log | sort
|
2017/12/16-16:37:08.315958 7f172affd700 [WARN] [db/column_family.cc:696] [local_28] Stalling writes because we have 20 level-0 files rate 16777216
|
2017/12/16-16:37:10.753871 7f16a9ffb700 [WARN] [db/column_family.cc:696] [local_28] Stalling writes because we have 20 level-0 files rate 16777216
|
2017/12/16-16:37:10.756102 7f16e9ffb700 [WARN] [db/column_family.cc:696] [local_28] Stalling writes because we have 21 level-0 files rate 16777216
|
2017/12/16-16:37:12.881411 7f16aa7fc700 [WARN] [db/column_family.cc:696] [local_28] Stalling writes because we have 21 level-0 files rate 16777216
|
2017/12/16-16:37:12.884727 7f17497fa700 [WARN] [db/column_family.cc:696] [local_28] Stalling writes because we have 22 level-0 files rate 16777216
|
..
|
Likely to be the main cause of the drops on OPS. From the RocksDB Documentation at https://github.com/facebook/rocksdb/wiki/Write-Stalls:
One thing worth to note is that slowdown/stop triggers and pending compaction bytes limit are per-column family, and write stalls apply to the whole DB, which means if one column family triggers write stall, the whole DB will be stalled
Attachments
Issue Links
- is triggering
-
MB-27326 [RocksDB] Merge the Local CF into the Seqno CF
-
- Closed
-