Metadata overhead warning is showing up while reading 2.1 billion deleted items. 100% cache miss ratio.

Steps:

1. Create a 3 node cluster
2. Create a magma bucket with 30GB RAM, 1 replica, Full eviction. 20 collections + 1 default
3. Create 2.1 billion items i.e. 100M in each collections. key size is 18, doc size is 256 bytes.
4. At this moment bucket is into 5.06638085714 DGM. Total Disk usage for kvstore is 1133567MB, wal is 1103MB, keyTree is 110612MB, seqTree is 1022716MB
5. Delete all the data from the bucket.
6. Read all the data to validate. During reads "Metadata overhead warning" is shown many times continuously every minute.

Over 107% of RAM allocated to bucket "GleamBook" on node "172.23.97.37" is taken up by keys and metadata.

Assumption:
With 100% cache miss ratio, seems like the delete keys+metadata is pulled from the storage and saved in the cache and not getting cleared from the cache anytime.

Test:

guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P "args=-i /tmp/dp_magma_job.ini -p bucket_storage=${storage},bucket_eviction_policy=fullEviction,rerun=False -t volumetests.Magma.volume.SteadyStateVolume,nodes_init=3,replicas=1,skip_cleanup=True,num_buckets=1,bucket_names=GleamBook,bucket_type=membase,compression_mode=off,iterations=1,batch_size=10,sdk_timeout=60,log_level=debug,infra_log_level=info,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,step_iterations=1,rollback=True,durability=None,fragmentation=50,rollbacks=10,num_collections=20,num_items=100000000,doc_size=256,pc=2,ramQuota=10240 -m rest"