Loading...

Details

Type: Bug
Resolution: Duplicate
Priority: Critical
Fix Version/s: None
Affects Version/s: 7.1.0
Component/s: couchbase-bucket, ns_server
Labels:
- eagle-eye
- system-test
Environment:
Enterprise Edition 7.1.0 build 1667

Triage:
Triaged
Operating System:
Centos 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.137.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.155.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.157.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.5.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.69.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.70.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.105.107.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.106.188.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.108.103.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.120.245.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.121.117.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.121.3.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.123.27.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.123.28.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.148.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.251.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.252.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.253.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.119.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.121.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.122.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.239.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.242.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.98.135.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.11.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.20.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.21.zip
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.25.zip

Show
         url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.137.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.155.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.157.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.5.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.69.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.104.70.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.105.107.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.106.188.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.108.103.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.120.245.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.121.117.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.121.3.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.123.27.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.123.28.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.148.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.251.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.252.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.96.253.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.119.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.121.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.122.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.239.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.97.242.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.98.135.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.11.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.20.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.21.zip          url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1636716491/collectinfo-2021-11-12T112814-ns_1%40172.23.99.25.zip
Story Points:
1
Is this a Regression?:
Unknown

Description

Test -
-test tests/integration/neo/test_neo_couchstore_milestone3.yml -scope tests/integration/neo/scope_couchstore.yml
Day - 3
Cycle - 2
Scale - 3

Test step -

[2021-11-12T01:37:55-08:00, sequoiatools/couchbase-cli:7.1:7387b4] server-add -c 172.23.108.103:8091 --server-add https://172.23.121.3 -u Administrator -p password --server-add-username Administrator --server-add-password password --services data

[2021-11-12T01:38:56-08:00, sequoiatools/couchbase-cli:7.1:f08293] failover -c 172.23.108.103:8091 --server-failover 172.23.106.100:8091 -u Administrator -p password --hard

[2021-11-12T01:39:10-08:00, sequoiatools/couchbase-cli:7.1:48a32f] rebalance -c 172.23.108.103:8091 -u Administrator -p password

REBALANCE FAILURE

2021-11-12T01:57:40.553-08:00, ns_rebalancer:2:info:message(ns_1@172.23.108.103) - Bad replicators after rebalance:

Missing = [{'ns_1@172.23.99.25','ns_1@172.23.104.157',765}]

Extras = []

2021-11-12T01:57:40.555-08:00, ns_orchestrator:0:critical:message(ns_1@172.23.108.103) - Rebalance exited with reason bad_replicas.

Rebalance Operation Id = 29fa18ff910639550efd9c153b05a1a0

At first glance this appears different from ~~MB-48631~~ as there is no chronicle crash involved.

Attachments

Issue Links

duplicates

MB-49542 OSO backfill can create orphaned checkpoint cursors for dead streams

Closed

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

[System Test] Rebalance exited with reason bad_replicas.

Details

Description

Attachments

Issue Links

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty