Ankit Prabhu diagnosed the issue on Slack
So its a race condition between undeployment and delete. When we click on delete in ui it will 1st delete the tempstore and then proceed to delete from primary store.
In primary store it will delete the settings path first
and then check whether the app is undergoing undeployment or not
Now since its undergoing undeployment it will simply return and it won't delete the primary store.
Now those checksome and definition remains in primary store
When we rebalance other nodes then it will try to read the settings from setting path and it won't be able to read and the interface will be nil
It will exit it by giving the interface conversion error.