When inserting documents in an extreme DGM scenario (1800 × 1MB worth of documents in a bucket with 400MB quota), less than 500 documents are inserted before no more progress is made.
This appears to be a regression somewhere between 6.0.0 and 6.5.0, still present on 7.0.0.
1. Create a two-node cluster, with 512MB data quota and all other services disabled
2. Create a bucket with 200MB memory quota, one replica, full eviction
/opt/couchbase/bin/cbc-pillowfight -U couchbase://localhost/lowmem -u Administrator -P password --num-items=1800 --set-pct=100 --min-size=1048575 --max-size=1048576 --timings --num-cycles=10 -v
800-1200 documents are inserted, pillowfight finishes within a minute
Less than 500 documents are inserted, pillowfight gets stuck. kv_mem_used_bytes goes slightly above kv_ep_mem_high_wat and stays there (even after terminating pillowfight), kv_dcp_backoff_replication climbs linearly, one node's CPU is pinned at 75% while another is nearly idle
7.0.0 cbcollects attached to this MB.