Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: 5.0.0, 5.1.0
Affects Version/s: 5.0.0
Component/s: couchbase-bucket
Labels:
- system-test

Triage:
Untriaged
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.106.188.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.107.47.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.108.107.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.237.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.238.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.239.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.242.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.20.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.21.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.22.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.25.zip
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.108.103.zip

Show
https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.106.188.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.107.47.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.108.107.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.237.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.238.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.239.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.97.242.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.20.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.21.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.22.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.99.25.zip https://s3.amazonaws.com/bugdb/jira/neg_res_ratio/collectinfo-2017-09-11T145645-ns_1%40172.23.108.103.zip
Is this a Regression?:
Yes

Description

Build : 5.0.0-3516

~~This is a regression from 5.0.0-3509 (RC4) since we did not see this issue in the system testing runs or elsewhere.~~ Currently unknown when this is a regression since (see comment from Mihir on 2017/09/13).

On the system testing longevity run with the build 5.0.0-3516 (2nd day of the run), we see some buckets reporting negative active resident ratio stats.

http://172.23.108.103:8091/ui/index.html#!/buckets/analytics/specific/@172.23.106.188:8091?statsHostname=all&bucket=DISTRICT&specificStat=vb_active_resident_items_ratio&openedStatsBlock=Server%20Resources&openedStatsBlock=Summary&openedSpecificStatsBlock=Specific%20Stats&zoom=minute (Administrator/password)

Same issue seen for buckets HISTORY and CUSTOMER.

Checked on the stats endpoint http://172.23.108.103:8091/pools/default/buckets/DISTRICT/stats, the following stats have negative values :

ep_resident_items_rate, vb_active_resident_items_ratio, vb_replica_resident_items_ratio

The cluster is live if needed for debugging.

Screenshot and logs attached.