Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-1938

low water mark ends up > max_size

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • 1.6.0 beta3
    • 1.6.0 beta4
    • couchbase-bucket
    • None
    • Operating System: All
      Platform: All

    Description

      If you add a bucket and then max_size gets reduced because you added a node or whatever, the high and low water marks don't seem to change, with the result that we never free written data from RAM. It appears that there's no random freeing yet?

      Here are stats with about 20k sets per second getting "out of memory" errors.

      ep_dbname: c:/Program Files (x86)/NorthScale/Server/data/ns_1/default
      ep_total_cache_size: 1321666825
      rejected_conns: 0
      ep_queue_age_cap: 900
      ep_commit_time: 0
      ep_oom_errors: 21890424
      ep_num_value_ejects: 1146709
      ep_bg_min_load: 15 ms
      cas_hits: 0
      curr_connections: 30
      ep_warmup_thread: complete
      auth_cmds: 132
      ep_max_data_size: 947912704
      ep_io_num_read: 940392
      ep_min_data_age: 0
      ep_bg_wait_avg: 228964270 s
      tap_mutation_received: 23887
      auth_errors: 0
      bytes_written: 3303706909
      ep_pending_ops: 0
      pointer_size: 32
      ep_dbinit: 10
      tap_mutation_sent: 48969
      ep_io_read_bytes: 1023894140
      mem_used: 1072700420
      ep_bg_load_avg: 69121289 s
      ep_pending_ops_max: 0
      ep_bg_max_wait: 18446744073 s
      version: 1.4.4_252_g4c4b211
      limit_maxbytes: 67108864
      ep_version: 1.6.0beta3a_17_g42dc800
      bytes_read: 24981572533
      ep_mem_high_wat: 1281097728
      ep_io_write_bytes: 264413290
      daemon_connections: 10
      cmd_get: 2183236
      ep_num_non_resident: 1140057
      ep_bg_fetched: 27640
      threads: 4
      curr_items: 1130296
      conn_yields: 44
      get_misses: 0
      ep_queue_size: 0
      delete_misses: 0
      ep_pending_ops_total: 0
      cmd_flush: 0
      connection_structures: 39
      ep_overhead: 12780068
      ep_max_txn_size: 50000
      ep_item_commit_failed: 0
      ep_too_young: 0
      total_connections: 181
      ep_bg_num_samples: 4270
      ep_kv_size: 1059920352
      ep_data_age: 44
      ep_bg_max_load: 18446744069 s
      cas_misses: 0
      ep_flusher_state: running
      incr_misses: 0
      ep_pending_ops_max_duration: 0 usec
      ep_total_persisted: 242581
      ep_storage_age_highwat: 54
      ep_io_num_write: 243752
      cmd_set: 22109120
      libevent: 1.4.13-stable
      ep_item_flush_failed: 0
      ep_mem_low_wat: 1024878182
      cas_badval: 0
      ep_flusher_todo: 0
      tap_vbucket_set_received: 42
      decr_misses: 0
      ep_warmup_time: 10
      ep_storage_type: featured
      pid: 3968
      tap_vbucket_set_sent: 146
      ep_warmed_up: 911728
      ep_warmup: true
      uptime: 2301
      bucket_conns: 19
      ep_flush_duration_highwat: 56
      decr_hits: 0
      incr_hits: 0
      ep_storage_age: 0
      ep_bg_min_wait: 15 ms
      ep_data_age_highwat: 55
      get_hits: 2183236
      ep_flush_duration: 0
      ep_too_old: 0
      ep_tap_keepalive: 0
      tap_connect_received: 92
      time: 1282353334
      ep_total_enqueued: 243752
      delete_hits: 0

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              dustin@sallings.org Dustin Sallings (Inactive)
              sean@northscale.com Sean Lynch (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty