Loading...

Details

Type: Bug
Resolution: Duplicate
Priority: Critical
Fix Version/s: 7.0.0
Affects Version/s: Cheshire-Cat
Component/s: storage-engine
Labels:

Triage:
Untriaged
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.106.134.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.106.136.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.58.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.73.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.74.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.75.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.77.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.81.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.86.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.121.77.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.24.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.25.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.26.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.31.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.32.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.33.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.122.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.14.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.243.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.254.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.48.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.105.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.110.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.112.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.148.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.149.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.150.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.151.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.241.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.74.zip

Show
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.106.134.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.106.136.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.58.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.73.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.74.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.75.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.77.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.81.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.120.86.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.121.77.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.24.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.25.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.26.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.31.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.32.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.123.33.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.122.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.14.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.243.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.254.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.96.48.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.105.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.110.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.112.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.148.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.149.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.150.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.151.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.241.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1621077422/collectinfo-2021-05-15T111703-ns_1%40172.23.97.74.zip
Story Points:
1
Is this a Regression?:
Unknown

Description

Build : 7.0.0-5161
Test : -test tests/2i/cheshirecat/test_idx_cc_vol_10K.yml -scope tests/2i/cheshirecat/scope_idx_cc_vol_10K.yml
Scale : 5
Iteration : 1st

A step in the Volume test to rebalance an index node after failover-recover-add back is stuck since 14 hours. Only one index idx2_aTPRH on bucket2.scope_6.collection_19 is in Moving state. The drain rate for this index is 0 and the num_docs_pending are 1323000 at the time of filing this report. See attached screenshot.

Logs were collected around 4 AM on 5/15.

[2021-05-14T22:07:43-07:00, sequoiatools/couchbase-cli:7.0:6b197d] failover -c 172.23.97.74:8091 --server-failover 172.23.120.86:8091 -u Administrator -p password --hard

[2021-05-14T22:07:59-07:00, sequoiatools/couchbase-cli:7.0:82c779] recovery -c 172.23.97.74:8091 --server-recovery 172.23.120.86:8091 --recovery-type full -u Administrator -p password

[2021-05-14T22:08:07-07:00, sequoiatools/couchbase-cli:7.0:9ef802] rebalance -c 172.23.97.74:8091 -u Administrator -p password

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

Screen Shot 2021-05-15 at 1.11.57 PM.png
176 kB
15/May/21 1:15 PM

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

[Volume Test] Memory difference between jemalloc usage stats and plasma usage stats

Details

Description

Attachments

Attachments

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty