Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47660

Writes stuck in extreme DGM scenario

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • Neo.next
    • 6.6.2, 6.5.0, 7.0.0, 7.1.0
    • clients
    • None
    • Two nodes, Ubuntu 20.04, 4 CPUs, 1GB RAM
    • Untriaged
    • Ubuntu 64-bit
    • 1
    • Yes
    • KV 2021-Oct-21, KV 2021-Nov

    Description

      Summary
      When inserting documents in an extreme DGM scenario (1800 × 1MB worth of documents in a bucket with 400MB quota), less than 500 documents are inserted before no more progress is made.

      This appears to be a regression somewhere between 6.0.0 and 6.5.0, still present on 7.0.0.

      Steps to Reproduce
      1. Create a two-node cluster, with 512MB data quota and all other services disabled
      2. Create a bucket with 200MB memory quota, one replica, full eviction

      /opt/couchbase/bin/cbc-pillowfight -U couchbase://localhost/lowmem -u Administrator -P password --num-items=1800 --set-pct=100 --min-size=1048575 --max-size=1048576 --timings --num-cycles=10 -v
      

      Expected Results
      800-1200 documents are inserted, pillowfight finishes within a minute

      Actual Results
      Less than 500 documents are inserted, pillowfight gets stuck. kv_mem_used_bytes goes slightly above kv_ep_mem_high_wat and stays there (even after terminating pillowfight), kv_dcp_backoff_replication climbs linearly, one node's CPU is pinned at 75% while another is nearly idle

      Artefacts
      7.0.0 cbcollects attached to this MB.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            marks.polakovs Marks Polakovs (Inactive)
            marks.polakovs Marks Polakovs (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty