Rebalance fails with unexpected_exit,badmatch,{error,closed} when memcached is killed and restarted

Steps:

1. initiate 4 nodes, load few documents 
2. swap rebalance 2 nodes (removing node ns_1@172.23.104.247 and rebalance afterwards, removing node ns_1@172.23.104.248 and rebalance afterwards)
3. stop and start memcached and restart rebalance
4. Rebalance fails with error

TestCase:

./testrunner -i /tmp/testexec.12568.ini num_items=10000,doc_size=512,transaction_timeout=200,durability=MAJORITY,atomicity=True,GROUP=P0_Set2,get-cbcollect-info=True -t rebalance_new.swaprebalancetests.SwapRebalanceFailedTests.test_failed_swap_rebalance,nodes_init=4,replicas=1,standard_buckets=4,num-swap=2,new_replica=3,swap-orchestrator=True,percentage_progress=30,GROUP=P0_Set2;default

<failure logs>:

u'text': u'Rebalance exited with reason {mover_crashed,\n {unexpected_exit,\n {\'EXIT\',<0.16547.3>,\n {badmatch,{error,closed,\n [{mc_client_binary,cmd_vocal_recv,5,\n [

{file,"src/mc_client_binary.erl"},\n {line,153}]},\n {mc_client_binary,\n wait_for_seqno_persistence,3,\n [{file,"src/mc_client_binary.erl"}

,\n {line,691}]},\n {ns_memcached,\n \'wait_for_seqno_persistence/3-fun-0\',3,\n [

{file,"src/ns_memcached.erl"},\n {line,1260}]},\n {ns_memcached,\n \'perform_very_long_call/3-fun-0\',2,\n [{file,"src/ns_memcached.erl"}

,\n {line,343}]},\n {ns_memcached_sockets_pool,\n \'executing_on_socket/3-fun-0\',3,\n [

{file,\n "src/ns_memcached_sockets_pool.erl"},\n {line,73}]},\n {async,\'async_init/4-fun-1\',3,\n [{file,"src/async.erl"},\{line,197}]}]},\n {gen_server,call,\n [{\'janitor_agent-bucket-1\',\n \'ns_1@172.23.104.251\'},\n \{if_rebalance,<0.10576.2>,\n {update_vbucket_state,95,active,\n undefined,undefined,undefined}},\n infinity]}}}}}.\nRebalance Operation Id = 141193d580851f808ee654572f10cfdb'}
2019-09-19 00:37:13,851 | test | ERROR | MainThread | [rest_client:print_UI_logs:2636] {u'code': 0, u'module': u'ns_vbucket_mover', u'type': u'critical', u'node': u'ns_1@172.23.104.247', u'tstamp': 1568878583770L, u'shortText': u'message', u'serverTime': u'2019-09-19T00:36:23.770Z', u'text': u'Worker <0.14953.3> (for action {move,{95,\n [\'ns_1@172.23.104.247\',\n \'ns_1@172.23.104.249\'],\n [\'ns_1@172.23.104.251\',\n \'ns_1@172.23.104.249\'],\n []}}) exited with reason {unexpected_exit,\n {\'EXIT\',\n <0.16547.3>,\n {badmatch,\n {error,\n closed,\n [{mc_client_binary,\n cmd_vocal_recv,\n 5,\n [{file,\n "src/mc_client_binary.erl"},\n {line,\n 153}]},\n {mc_client_binary,\n wait_for_seqno_persistence,\n 3,\n [{file,n "src/mc_client_binary.erl"},\n \{line,\n 691}]},\n {ns_memcached,\n \'wait_for_seqno_persistence/3-fun-0\',\n 3,\n [{file,\n "src/ns_memcached.erl"},\n {line,\n 1260}]},\n {ns_memcached,\n \'perform_very_long_call/3-fun-0\',\n 2,\n [{file,n "src/ns_memcached.erl"},\n \{line,\n 343}]},\n {ns_memcached_sockets_pool,\n \'executing_on_socket/3-fun-0\',\n 3,\n [{file,n "src/ns_memcached_sockets_pool.erl"}

,\n {line,\n 73}]},\n {async,\n \'async_init/4-fun-1\',\n 3,\n [

{file,\n "src/async.erl"}

,\n {line,\n 197}]}]},\n {gen_server,\n call,\n [

{\'janitor_agent-bucket-1\',\n \'ns_1@172.23.104.251\'}

,\n {if_rebalance,\n <0.10576.2>,\n {update_vbucket_state,\n 95,\n active,\n undefined,\n undefined,\n undefined}},\n infinity]}}}}'}