memcached exits with status 134 and rebalance failures in centos longevity

Service 'memcached' exited with status 134. Restarting. Messages:

2018-03-01T11:51:50.044483Z CRITICAL /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f4a1da74000+0x8f221]

2018-03-01T11:51:50.044496Z CRITICAL /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f4a1da74000+0x8ff6f]

2018-03-01T11:51:50.044502Z CRITICAL /opt/couchbase/bin/memcached() [0x400000+0x5d72e]

2018-03-01T11:51:50.044507Z CRITICAL /opt/couchbase/bin/memcached() [0x400000+0x3cc51]

2018-03-01T11:51:50.044513Z CRITICAL /opt/couchbase/bin/../lib/libevent_core.so.2.1.8() [0x7f4a1e8c3000+0x1937c]

2018-03-01T11:51:50.044517Z CRITICAL /opt/couchbase/bin/../lib/libevent_core.so.2.1.8(event_base_loop+0x46f) [0x7f4a1e8c3000+0x1c7cf]

2018-03-01T11:51:50.044522Z CRITICAL /opt/couchbase/bin/memcached() [0x400000+0x3b2b4]

2018-03-01T11:51:50.044526Z CRITICAL /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0() [0x7f4a1f7ba000+0x8387]

2018-03-01T11:51:50.044531Z CRITICAL /lib64/libpthread.so.0() [0x7f4a1f17a000+0x7dc5]

2018-03-01T11:51:50.044556Z CRITICAL /lib64/libc.so.6(clone+0x6d) [0x7f4a1d19b000+0xf776d] hide

Service 'memcached' exited with status 134. Restarting. Messages:

2018-03-01T12:02:02.435045Z WARNING (default) DCP (Producer) eq_dcpq:xdcr:dcp_f4268e62702130298bf87f17cc481219/default/default_172.23.97.239:11210_0:grt1Bjtc7yTjp5yVwpfndg== - (vb 800) Stream request requires rollback to seqno:0 because consumer ahead of producer - producer upper at 23323. Client requested seqnos:{21750,18446744073709551615} snapshot:{0,40409} uuid:241399521537618

2018-03-01T12:02:02.443382Z WARNING (default) DCP (Producer) eq_dcpq:xdcr:dcp_f4268e62702130298bf87f17cc481219/default/default_172.23.97.239:11210_0:grt1Bjtc7yTjp5yVwpfndg== - (vb 800) Stream request requires rollback to seqno:0 because consumer ahead of producer - producer upper at 23323. Client requested seqnos:{16841,18446744073709551615} snapshot:{0,39309} uuid:241399521537618

2018-03-01T12:02:02.451255Z WARNING (default) DCP (Producer) eq_dcpq:xdcr:dcp_f4268e62702130298bf87f17cc481219/default/default_172.23.97.239:11210_0:grt1Bjtc7yTjp5yVwpfndg== - (vb 800) Stream request requires rollback to seqno:0 because consumer ahead of producer - producer upper at 23323. Client requested seqnos:{10188,18446744073709551615} snapshot:{0,39297} uuid:241399521537618 hide

Rebalance exited with reason {mover_crashed,

{unexpected_exit,

{'EXIT',<0.6172.206>,

{bulk_set_vbucket_state_failed,

[{'ns_1@172.23.96.56',

{'EXIT',

{{{{{case_clause,

{error,

{{{badmatch,

{error,

{{badmatch,{error,econnrefused}},

[{dcp_proxy,connect,5,

[{file,"src/dcp_proxy.erl"},

{line,228}]},

{dcp_proxy,maybe_connect,2,

[{file,"src/dcp_proxy.erl"},

{line,210}]},

{dcp_producer_conn,init,2,

[{file,

"src/dcp_producer_conn.erl"},

{line,31}]},

{dcp_proxy,init,1,

[{file,"src/dcp_proxy.erl"},

{line,57}]},

{gen_server,init_it,6,

[{file,"gen_server.erl"},

{line,304}]},

{proc_lib,init_p_do_apply,3,

[{file,"proc_lib.erl"},

{line,239}]}]}}},

[{dcp_replicator,init,1,

[{file,"src/dcp_replicator.erl"},

{line,50}]},

{gen_server,init_it,6,

[{file,"gen_server.erl"},

{line,304}]},

{proc_lib,init_p_do_apply,3,

[{file,"proc_lib.erl"},

{line,239}]}]},

{child,undefined,

{'ns_1@172.23.97.239',true},

{dcp_replicator,start_link,

['ns_1@172.23.97.239',"STOCK",

true]},

temporary,60000,worker,

[dcp_replicator]}}}},

[{dcp_sup,start_replicator,2,

[{file,"src/dcp_sup.erl"},

{line,54}]},

{dcp_sup,

'-manage_replicators/3-lc$^3/1-3-',

2,

[{file,"src/dcp_sup.erl"},

{line,81}]},

{dcp_replication_manager,

handle_call,3,

[{file,

"src/dcp_replication_manager.erl"},

{line,87}]},

{gen_server,handle_msg,5,

[{file,"gen_server.erl"},

{line,585}]},

{proc_lib,init_p_do_apply,3,

[{file,"proc_lib.erl"},

{line,239}]}]},

{gen_server,call,

['dcp_replication_manager-STOCK',

{manage_replicators,

['ns_1@172.23.108.103',

'ns_1@172.23.97.237',

'ns_1@172.23.97.238',

'ns_1@172.23.97.239'],

true},

infinity]}},

{gen_server,call,

['replication_manager-STOCK',

{change_vbucket_replication,779,

'ns_1@172.23.97.239'},

infinity]}},

{gen_server,call,

[{'janitor_agent-STOCK',

'ns_1@172.23.96.56'},

{if_rebalance,<0.10452.205>,

{update_vbucket_state,779,replica,

undefined,'ns_1@172.23.97.239'}},

infinity]}}}}]}}}} hide

ns_orchestrator 000

ns_1@172.23.108.103

12:02:03 PM   Thu Mar 1, 2018

<0.930.206> exited with {unexpected_exit,

{'EXIT',<0.6172.206>,

{bulk_set_vbucket_state_failed,

[{'ns_1@172.23.96.56',

{'EXIT',

{{{{{case_clause,

{error,

{{{badmatch,

{error,

{{badmatch,{error,econnrefused}},

[{dcp_proxy,connect,5,

[{file,"src/dcp_proxy.erl"},

{line,228}]},

{dcp_proxy,maybe_connect,2,

[{file,"src/dcp_proxy.erl"},

{line,210}]},

{dcp_producer_conn,init,2,

...show