For example I would selfhost Mastodon with me@fedi.com but then decide to change the server software to GoToSocial but keep the same me@fedi.com. Would everyone who was following me@fedi.com keep me in their follower list or would whole fediverse crash?
It definitely depends on the software, for example in matrix you can easily cause federation issues in some cases. this won’t blow up the fediverse, but it could impact the user experience if they rely on or interact with that instance.