Details
-
Bug
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
Untriaged
-
Unknown
Description
Build 7.0.0 build 1655
Steps to reproduce. This is not consistent but found this issue when working the below steps.
- Cluster with kv,kv,kv,kv,search,search
- Create default bucket
- Create default index with 2 fields
- Load bucket with 100M docs each of size 50B
- Let index complete
- Now start loading another 200M new docs while mutating 100M docs loaded before.
- And in parallel run index defn update, updating replica count to 0, sleep 5mins and update it back to 1
- And when the indexing is at around 55%. Stopped the above step.
- Index was not completed even after 2 days, instead it was stuck at 95.14%.
Logs:
https://cb-jira.s3.us-east-2.amazonaws.com/logs/index-stuck/collectinfo-2020-03-30T021332-ns_1%40172.23.121.195.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/index-stuck/collectinfo-2020-03-30T021332-ns_1%40172.23.121.249.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/index-stuck/collectinfo-2020-03-30T021332-ns_1%40172.23.122.185.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/index-stuck/collectinfo-2020-03-30T021332-ns_1%40172.23.122.187.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/index-stuck/collectinfo-2020-03-30T021332-ns_1%40172.23.122.188.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/index-stuck/collectinfo-2020-03-30T021332-ns_1%40172.23.122.190.zip
In this instance, step 7 was stopped at 2020-03-27T18:34:59. Index was stuck at 95.14% even after 2 days : Sun Mar 29 19:38:51 PDT 2020.