Runbook for validating indexer node is not undersized

We should cover how to check if indexer is undersized in run book.   For this we need to know if indexes are 0 resident ratio because memory quota is too low or index not being used at all. 

From slack
Hi Team,
From a customer's cluster, I'm observing that the resident ratio of the indexes is 0%, does it indicate the cluster is undersized?
 
you can check in indexer.log for these warnings Warning: not enough memory to hold records in memory  which we see in general if memory quota is low. You can also check if there is a possibility of a restart.
 
you can also check the index node level stat in indexer_stats.log avg_resident_percent . This gives a better picture from sizing perspective than looking at the individual index level stat. If an index is not being used or indexer restarts, index resident ratio can be low. You can also double check it with total_data_size node level stat and get a rough idea how much memory would be required at 10% RR.
 
Also, the effective memory quota will be reported as aggregated stat to prometheus starting 7.6 - storage_stats_manager_enterprise.go