Loading...

Details

Type: Bug
Resolution: Duplicate
Priority: Critical
Fix Version/s: 7.1.0
Affects Version/s: 7.1.0
Component/s: query
Labels:

Triage:
Untriaged
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.58.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.73.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.74.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.75.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.77.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.81.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.86.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.121.77.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.24.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.25.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.26.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.31.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.32.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.33.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.96.122.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.96.14.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.96.48.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.97.112.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.97.241.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.97.74.zip

Show
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.58.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.73.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.74.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.75.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.77.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.81.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.120.86.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.121.77.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.24.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.25.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.26.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.31.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.32.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.123.33.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.96.122.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.96.14.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.96.48.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.97.112.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.97.241.zip url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1634641361/collectinfo-2021-10-19T110243-ns_1%40172.23.97.74.zip
Story Points:
1
Is this a Regression?:
Unknown

Description

Build : 7.1.0-1524
Test : -test tests/integration/neo/test_neo_magma_milestone2.yml -scope tests/integration/neo/scope_neo_magma.yml
Scale : 3
Iteration : 1st

This is the longevity test with all buckets having Magma storage backend. cbq-engine is crashing on both the query nodes in the cluster - 172.23.96.254 & 172.23.97.112 due to being OOM killed.

172.23.96.254 : crash

[user:info,2021-10-19T02:55:46.324-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

[user:info,2021-10-19T03:03:03.853-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

[user:info,2021-10-19T03:07:07.204-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

[user:info,2021-10-19T03:34:00.169-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

[user:info,2021-10-19T03:38:57.391-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

[user:info,2021-10-19T03:46:26.313-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

[user:info,2021-10-19T03:51:41.871-07:00,ns_1@172.23.96.254:<0.2592.1>:ns_log:consume_log:76]Service 'query' exited with status 137. Restarting. Messages:

172.23.97.112 : crash

[user:info,2021-10-19T02:47:39.462-07:00,ns_1@172.23.97.112:<0.31954.0>:ns_log:consume_log:76]Service 'query' exited with status 2. Restarting. Messages:

172.23.96.254 : query

	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.6/go/src/runtime/panic.go:1117 +0x72

172.23.97.112 : query

	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.16.6/go/src/runtime/panic.go:1117 +0x72

172.23.96.254 : query

fatal error: runtime: out of memory

fatal error: runtime: out of memory

172.23.97.112 : query

fatal error: runtime: out of memory

Logs attached were collected around 4 AM PST on 10/19/21.

Attachments

Issue Links

duplicates

MB-48996 Builds are unexpectedly ASan builds

Closed

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

[System Test] cbq-engine OOM killed multiple times

Details

Description

Attachments

Issue Links

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty