Details
-
Bug
-
Resolution: Fixed
-
Critical
-
5.5.0
-
Untriaged
-
-
Unknown
Description
Following query service exit found in 5.5.0-1911 centos longevity:
172.23.108.103 diag.log:
2018-02-20T19:21:42.428-08:00, ns_log:0:info:message(ns_1@172.23.99.25) - Service 'query' exited with status 137. Restarting. Messages:
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:57.539-08:00 [Info] connected with 3 indexers
|
2018-02-20T19:19:57.569-08:00 [Info] GSIC[default/default-1519163795502456181] logstats "default" {"gsi_scan_count":186240,"gsi_scan_duration":8226023493604998,"gsi_throttle_duration":5910862131355,"gsi_prime_duration":4605593043229069,"gsi_blocked_duration":717600393832501,"gsi_totalbackfills":177540}
|
2018-02-20T19:19:58.121-08:00 [Info] GSIC[default/CUSTOMER-1519163860497638031] logstats "CUSTOMER" {"gsi_scan_count":97445,"gsi_scan_duration":11787660215481736,"gsi_throttle_duration":0,"gsi_prime_duration":298260055850031,"gsi_blocked_duration":0,"gsi_totalbackfills":0}
|
[goport(/opt/couchbase/bin/cbq-engine)] 2018/02/20 19:21:39 child process exited with status 137
|
|
-------------------------------
|
|
|
per_node_processes('ns_1@172.23.108.103') =
|
{<0.32765.73>,
|
[{registered_name,'dcp_consumer_conn-WAREHOUSE-ns_1@172.23.108.104'},
|
{status,waiting},
|
{initial_call,{proc_lib,init_p,5}},
|
{backtrace,[<<"Program counter: 0x00007f989f9c9920 (gen_server:loop/6 + 264)">>,
|
<<"CP: 0x0000000000000000 (invalid)">>,<<"arity = 0">>,
|
<<>>,
|
<<"0x00007f9851d4af68 Return addr 0x00007f989f8f0210 (proc_lib:init_p_do_apply/3 + 56)">>,
|
<<"y(0) []">>,<<"y(1) 10000">>,
|
<<"y(2) dcp_proxy">>,
|
<<"(3) {state,#Port<0.124902>,{consumer,\"replication:ns_1@172.23.108.104->ns_1@172.23.108.103:WAREHOUSE\",'ns_1@172.23.1">>,
|
<<"y(4) <0.32765.73>">>,<<"y(5) <0.32696.73>">>,
|
<<>>,
|
<<"0x00007f9851d4afa0 Return addr 0x0000000000891848 (<terminate process normally>)">>,
|
<<"y(0) Catch 0x00007f989f8f0230 (proc_lib:init_p_do_apply/3 + 88)">>,
|
<<>>]},
|
{error_handler,error_handler},
|
{garbage_collection,[{min_bin_vheap_size,46422},
|
{min_heap_size,233},
|
{fullsweep_after,512},
|
{minor_gcs,152}]},
|
{heap_size,1598},
|
{total_heap_size,2585},
|
{links,[<0.32696.73>,#Port<0.124902>]},
|
{monitors,[]},
|
{monitored_by,[<0.6289.0>]},
|
{memory,21632},
|
{messages,[]},
|
{message_queue_len,0},
|
{reductions,7882112},
|
{trap_exit,false},
|
{current_location,{gen_server,loop,6,
|
[{file,"gen_server.erl"},{line,358}]}},
|
{dictionary,[{'$ancestors',['dcp_replicator-WAREHOUSE-ns_1@172.23.108.104',
|
'dcp_sup-WAREHOUSE',
|
'single_bucket_kv_sup-WAREHOUSE',
|
ns_bucket_sup,ns_bucket_worker_sup,
|
ns_server_sup,ns_server_nodes_sup,
|
<0.170.0>,ns_server_cluster_sup,<0.89.0>]},
|
{'$initial_call',{dcp_proxy,init,1}}]}]}
|
172.23.99.25 - query log:
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:54.291-08:00 [INFO] Using plain authentication for user @cbq-engine
|
2018-02-20T19:19:57.539-08:00 [Info] connected with 3 indexers
|
2018-02-20T19:19:57.569-08:00 [Info] GSIC[default/default-1519163795502456181] logstats "default" {"gsi_scan_count":186240,"gsi_scan_duration":8226023493604998,"gsi_throttle_duration":5910862131355,"gsi_prime_duration":4605593043229069,"gsi_blocked_duration":717600393832501,"gsi_totalbackfills":177540}
|
2018-02-20T19:19:58.121-08:00 [Info] GSIC[default/CUSTOMER-1519163860497638031] logstats "CUSTOMER" {"gsi_scan_count":97445,"gsi_scan_duration":11787660215481736,"gsi_throttle_duration":0,"gsi_prime_duration":298260055850031,"gsi_blocked_duration":0,"gsi_totalbackfills":0}
|
[goport(/opt/couchbase/bin/cbq-engine)] 2018/02/20 19:21:39 child process exited with status 137
|
_time=2018-02-20T19:21:58.173-08:00 _level=INFO _msg= Initialization of cbauth succeeded
|
_time=2018-02-20T19:21:59.303-08:00 _level=INFO _msg=New store created with url http://127.0.0.1:8091
|
_time=2018-02-20T19:21:59.303-08:00 _level=INFO _msg=pollEOF: About to start stdin polling
|
_time=2018-02-20T19:21:59.305-08:00 _level=INFO _msg=cbq-engine started servicers=32 plus-servicers=128 timeout=0s max-index-api=3 n1ql_feat_cntrl=0 version=2.0.0-N1QL datastore=http://127.0.0.1:8091 scan-cap=512 pipeline-cap=512 pipeline-batch=16 request-cap=256 max-concurrency=8 loglevel=INFO request-size-cap=67108864
|
_time=2018-02-20T19:21:59.307-08:00 _level=INFO _msg=HttpEndpoint: Listen Address=[::]:8093
|
_time=2018-02-20T19:21:59.307-08:00 _level=INFO _msg= Certificates have been refreshed by ns server
|
_time=2018-02-20T19:21:59.329-08:00 _level=INFO _msg=New settings received: {"query.settings.curl_whitelist":{"all_access":false,"allowed_urls":[],"disallowed_urls":[]},"query.settings.tmp_space_dir":"/opt/couchbase/var/lib/couchbase/tmp","query.settings.tmp_space_size":5120}
|
|
_time=2018-02-20T19:21:59.336-08:00 _level=INFO _msg=HttpEndpoint: ListenTLS Address=[::]:18093
|
_time=2018-02-20T19:21:59.437-08:00 _level=INFO _msg=ERROR: Starting TLS listener - listen tcp :18093: bind: address already in use
|
_time=2018-02-20T19:21:59.483-08:00 _level=INFO _msg=Settings notifier from metakv
|
|
_time=2018-02-20T19:21:59.483-08:00 _level=INFO _msg=New Value for curl whitelist <ud>map[disallowed_urls:[] all_access:false allowed_urls:[]]</ud>
|
_time=2018-02-20T19:21:59.503-08:00 _level=INFO _msg=Created New Bucket default
|
_time=2018-02-20T19:22:00.307-08:00 _level=INFO _msg=Created New Bucket CUSTOMER
|
2018-02-20T19:22:00.385-08:00 [Info] GSI client: removing old file /opt/couchbase/var/lib/couchbase/tmp/scan-results25882003408218 last-modified @ 2018-02-20 14:33:05.192534865 -0800 PST
|
2018-02-20T19:22:00.387-08:00 [Info] New settings received:
|
Attachments
Issue Links
- relates to
-
MB-28710 Rebalance exited with reason {badmatch, {leader_activities_error, {default,rebalance}, quorum_lost}}
- Closed