ep-engine does not flush expired replica items if expiry time is 2-3 seconds (memory leak)

to reproduce :

add 2k items with expiry = 0
then add 10k items each 50k where expiry is set to 2 seconds
wait until expiry pager runs( you can modify the expiry pager time to 120 seconds instead of the default which is 60 minutes)

then look at the active and replica items:

in my case : active items : 2000 , replica items on node 1: 6700 and node2: 7800 items

i used mbstats hash to get the number of items in acrive and replica vbucket :

vb_38 is active on node 10.1.5.38 and replica in 10.1.5.54

vb_38:counted is 1 for the active node versus 16 on the replica

root@ubuntu1004-64:~# /opt/membase/bin/mbstats 10.1.5.54:11210 hash detail | grep vb_38:
vb_38:counted: 16
vb_38:histo_0,1: 3063
vb_38:histo_1,2: 16
vb_38:locks: 5
vb_38:max_depth: 1
vb_38:mem_size: 2288
vb_38:mem_size_counted: 2288
vb_38:min_depth: 0
vb_38:reported: 16
vb_38:resized: 0
vb_38:size: 3079
vb_38:state: replica
root@ubuntu1004-64:~# /opt/membase/bin/mbstats 10.1.5.38:11210 hash detail | grep vb_38:
vb_38:counted: 1
vb_38:histo_0,1: 3078
vb_38:histo_1,2: 1
vb_38:locks: 5
vb_38:max_depth: 1
vb_38:mem_size: 143
vb_38:mem_size_counted: 143
vb_38:min_depth: 0
vb_38:reported: 1
vb_38:resized: 0
vb_38:size: 3079
vb_38:state: active
root@ubuntu1004-64:~#

because of this the memory usage also remains higher than it should be :

32613 membase 20 0 1248m 1.1g 3088 S 2 45.8 2:43.53 memcached

root@ubuntu1004-64:~# /opt/membase/bin/mbstats 10.1.5.54:11210 all | grep mem_
ep_mem_high_wat: 943718400
ep_mem_low_wat: 754974720
mem_used: 27999326

mem_used is 279MB
if i restart the node mem_used drops to 60 meg and memcached resident memory also becomes 70 MB ( down from 1.1 GB )

after restart active and replica items count will also drop to 1000