That isn’t what that document says. It says that they can impersonate you in non-E2EE scenarios. The clients I use warn me when a message isn’t properly encrypted so someone without E2EE keys can’t impersonate someone in an E2EE room.

That being said the general concept is a problem. I would love to see progress where all events from a user are signed by a device key and non-forgable. There is some thinking about this with portable identities (such as MSC2787) where you server is basically just storing and forwarding events but the root of trust is your identity and keys that you control. But none of this will land soon, not for many years.