Details
-
Bug
-
Resolution: Fixed
-
1.6.0 beta1
-
None
-
Operating System: All
Platform: All
Description
From Sharon:
I run load on a 3 node cluster via moxi, and removed one of the nodes. Moxi
mis-behave buy stop sending traffic, but that is a different problem.
Bucket is configured to 2G, load was on 10K items
/opt/memcachetest/bin/memcachetest.old -h 10.2.1.11 -s123456 -t 32 -i 10000 -c
500000 –l
I started getting many missing data errors:
Failed to set data!
missing data for <00001912>
Checked the stats
STAT mem_used 18446744073489040812
STAT ep_overhead 18446744073390441760
STAT ep_max_data_size 2097152000
[root@Config156VM0 ~]# telnet 10.2.1.14 11211
Trying 10.2.1.14...
Connected to 10.2.1.14 (10.2.1.14).
Escape character is '^]'.
stats
STAT delete_misses 0
STAT ep_io_num_write 12384448
STAT rejected_conns 0
STAT connection_structures 116
STAT limit_maxbytes 67108864
STAT decr_hits 0
STAT ep_bg_load_avg 42 usec
STAT ep_bg_wait_avg 175 ms
STAT ep_too_young 0
STAT curr_connections 115
STAT rusage_system 181.567398
STAT ep_io_write_bytes 25453983895
STAT ep_total_cache_size 141770806
STAT ep_storage_age 180
STAT ep_flush_duration_highwat 70
STAT ep_flush_duration 6
STAT cas_misses 0
STAT ep_flusher_todo 0
STAT tap_mutation_received 743656
STAT mem_used 18446744073489040812
STAT tap_mutation_sent 737787
STAT get_misses 2512
STAT ep_num_value_ejects 46908
STAT ep_queue_size 1990
STAT bytes_read 3190859489
STAT get_hits 1475634
STAT tap_vbucket_set_received 3504
STAT decr_misses 0
STAT ep_bg_min_wait 9 usec
STAT rusage_user 352.506409
STAT bucket_conns 81
STAT ep_num_non_resident 31027
STAT ep_tap_keepalive 0
STAT ep_too_old 9926244
STAT cmd_flush 0
STAT ep_bg_max_load 3307 usec
STAT ep_max_txn_size 100000
STAT ep_version 3.200000
STAT uptime 8003
STAT ep_data_age_highwat 6137
STAT ep_queue_age_cap 1800
STAT incr_hits 0
STAT time 1281886658
STAT ep_total_persisted 782410
STAT daemon_connections 20
STAT ep_flusher_state running
STAT pointer_size 64
STAT version 1.400000
STAT ep_max_data_size 2097152000
STAT ep_warmup_time 0
STAT ep_item_commit_failed 0
STAT total_connections 3140
STAT curr_items 16551
STAT ep_data_age 32
STAT delete_hits 0
STAT ep_bg_max_wait 1242 ms
STAT ep_storage_type featured
STAT ep_total_enqueued 807233
STAT ep_mem_low_wat 1258291200
STAT ep_kv_size 98599052
STAT ep_min_data_age 0
STAT ep_io_num_read 53341
STAT ep_warmed_up 9670
STAT ep_item_flush_failed 11584491
STAT cas_hits 0
STAT ep_warmup true
STAT ep_dbname 0.000000
STAT ep_commit_time 3
STAT auth_errors 0
STAT ep_bg_fetched 41623
STAT ep_storage_age_highwat 6316
STAT threads 8
STAT pid 13686
STAT auth_cmds 2406
STAT cas_badval 0
STAT cmd_set 768280
STAT ep_io_read_bytes 105430814
STAT cmd_get 1478146
STAT tap_vbucket_set_sent 2432
STAT conn_yields 1425
STAT ep_warmup_thread complete
STAT tap_connect_received 1243
STAT bytes_written 3199501268
STAT libevent 2.800000
STAT ep_bg_num_samples 41623
STAT ep_mem_high_wat 1572864000
STAT ep_dbinit 0
STAT incr_misses 0
STAT ep_bg_min_load 25 usec
STAT ep_overhead 18446744073390441760
END
Attachments
Issue Links
- is duplicated by
-
MB-1839 moxi stats (mem_used) too huge in a cluster
- Closed