Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
7.6.0
-
7.6.0-1500
-
Untriaged
-
Linux arm64
-
-
0
-
Unknown
Description
Steps:
- Installed cb-server 7.6.0-1500
- Enable n2n encryption (all)
- Create 5 node KV nodes cluster
- Create 'Magma' bucket
Observation:
Seeing janitor_agent crashing continuously for the created bucket and never recovers back
[error_logger:error,2023-09-14T03:24:28.647-07:00,ns_1@172.23.110.64:janitor_agent-bucket-0<0.3195.6>:ale_error_logger_handler:do_log:101]
|
=========================CRASH REPORT=========================
|
crasher:
|
initial call: janitor_agent:init/1
|
pid: <0.3195.6>
|
registered_name: 'janitor_agent-bucket-0'
|
exception exit: {{badarg,
|
[{ejson,reverse_tokens,[undefined],[]},
|
{ejson,nif_decode,1,
|
[{file,
|
"/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/ejson/ejson.erl"},
|
{line,40}]},
|
{ns_memcached,'-get_bucket_stats/3-fun-1-',4,
|
[{file,"src/ns_memcached.erl"},{line,1806}]},
|
{ns_memcached,'-perform_very_long_call/3-fun-0-',2,
|
[{file,"src/ns_memcached.erl"},{line,404}]},
|
{ns_memcached_sockets_pool,
|
'-executing_on_socket/3-fun-0-',3,
|
[{file,"src/ns_memcached_sockets_pool.erl"},
|
{line,88}]},
|
{async,'-async_init/4-fun-1-',3,
|
[{file,"src/async.erl"},{line,199}]}]},
|
{gen_server,call,
|
[{'ns_memcached-bucket-0','ns_1@172.23.110.64'},
|
{raw_stats,<<"dcp">>,
|
#Fun<dcp_replicator.3.103011606>,[]},
|
300000]}}
|
in function gen_server:call/3 (gen_server.erl, line 385)
|
in call from ns_memcached:do_call/4 (src/ns_memcached.erl, line 1624)
|
in call from dcp_replicator:get_connections/1 (src/dcp_replicator.erl, line 296)
|
in call from dcp_sup:nuke/1 (src/dcp_sup.erl, line 114)
|
in call from janitor_agent:init/1 (src/janitor_agent.erl, line 591)
|
ancestors: ['janitor_agent_sup-bucket-0',
|
'single_bucket_kv_sup-bucket-0',ns_bucket_sup,
|
ns_bucket_worker_sup,ns_server_sup,ns_server_nodes_sup,
|
<0.291.0>,ns_server_cluster_sup,root_sup,<0.155.0>]
|
message_queue_len: 0
|
messages: []
|
links: [<0.3131.6>]
|
dictionary: []
|
trap_exit: false
|
[ns_server:debug,2023-09-14T03:24:28.664-07:00,ns_1@172.23.110.64:<0.3203.6>:ns_memcached_sockets_pool:executing_on_socket:92]Exception while executing on socket {{127,0,0,1},58216}: {error,badarg, [{ejson, reverse_tokens, [undefined], []}, {ejson,nif_decode, 1, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/ejson/ejson.erl"}, {line,40}]}, {ns_memcached, '-get_bucket_stats/3-fun-1-', 4, [{file, "src/ns_memcached.erl"}, {line,1806}]}, {ns_memcached, '-perform_very_long_call/3-fun-0-', 2, [{file, "src/ns_memcached.erl"}, {line,404}]}, {ns_memcached_sockets_pool, '-executing_on_socket/3-fun-0-', 3, [{file, "src/ns_memcached_sockets_pool.erl"}, {line,88}]}, {async, '-async_init/4-fun-1-', 3, [{file, "src/async.erl"}, {line,199}]}]}[ns_server:debug,2023-09-14T03:24:28.665-07:00,ns_1@172.23.110.64:ns_memcached-bucket-0<0.3198.6>:ns_memcached:terminate:929]Dying when socket is not yet connected
|
[error_logger:error,2023-09-14T03:24:28.665-07:00,ns_1@172.23.110.64:ns_memcached-bucket-0<0.3198.6>:ale_error_logger_handler:do_log:101] =========================ERROR REPORT========================= ** Generic server 'ns_memcached-bucket-0' terminating ** Last message in was {connect_done,4,{ok,#Port<0.49118>}} ** When Server state == {state,4,0,0, {[{{raw_stats,<<"dcp">>, #Fun<dcp_replicator.3.103011606>,[]}, {<0.3201.6>, [alias|#Ref<0.3325611495.4024762376.135942>]}, {1694,687068,648827}, 2}], []}, {[],[]}, {[],[]}, connecting,undefined,"bucket-0", [collections,json], [],still_connecting,[],[],<0.3199.6>,false,10000} ** Reason for termination == ** {badarg, [{ejson,reverse_tokens,[undefined],[]}, {ejson,nif_decode,1, [{file, "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/ejson/ejson.erl"}, {line,40}]}, {ns_memcached,'-get_bucket_stats/3-fun-1-',4, [{file,"src/ns_memcached.erl"},{line,1806}]}, {ns_memcached,'-perform_very_long_call/3-fun-0-',2, [{file,"src/ns_memcached.erl"},{line,404}]}, {ns_memcached_sockets_pool,'-executing_on_socket/3-fun-0-',3,
|
Note: Have seen this from different runs since 7.6.0-1440 but not reproducible consistently
Attachments
Issue Links
- duplicates
-
MB-58654 [AWS]: Bucket is stuck in warmup immediately after creation.
- Closed