Rebalance failed with "Service 'index' exited with status 2"

{'node': 'ns_1@10.123.230.101', 'type': 'critical', 'code': 0, 'module': 'ns_orchestrator', 'tstamp': 1675335907202, 'shortText': 'message', 'text': "Rebalance exited with reason {service_rebalance_failed,index,\n                              {agent_died,<34608.17097.0>,\n                               {lost_connection,\n                                {'ns_1@10.123.230.102',shutdown}}}}.\nRebalance Operation Id = 35a43cfc3cec18ead9c75f80aab47ba0", 'serverTime': '2023-02-02T06:05:07.202Z'}

2023-02-02 16:35:08 | ERROR | MainProcess | Cluster_Thread | [on_prem_rest_client.print_UI_logs] {'node': 'ns_1@10.123.230.102', 'type': 'info', 'code': 0, 'module': 'ns_log', 'tstamp': 1675335905969, 'shortText': 'message', 'text': "Service 'index' exited with status 2. Restarting. Messages:\ngithub.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).insertSecIndex(0xc005c58000, {0xc007a4bc20, 0x1c7, 0x1e0}, {0xc00bf08750, 0x14, 0x18}, 0x0, 0x1, 0xc0074b3a00)\n\t/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:1009 +0x44a\ngithub.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).insert(0xc005c58000, {0xc007a4bc20?, 0x0?, 0xc0056cec38?}, {0xc00bf08750?, 0xc0056cec20?, 0x1c2f1c0?}, 0xc007505868?, 0x40?, 0xc0074b3a00)\n\t/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:943 +0xb9\ngithub.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).handleCommandsWorker(0xc005c58000, 0x0)\n\t/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:816 +0x425\ncreated by github.com/couchbase/indexing/secondary/indexer.(*plasmaSlice).initWriters\n\t/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/plasma_slice.go:3708 +0x3cd\n", 'serverTime': '2023-02-02T06:05:05.969Z'}