For anyone who stumbles on this, I’ve been using a pretty solid solution. These features are available thanks to this 2021 discussion that I missed: Internet Identity Lack Of Security
You can create a seed phrase and then disable the changing of it without the existing seed phrase. (I didn’t understand this. I guess my seed phrase is now just between me and the NSA).
Then add neurons and high value tokens inside the Ledger wallet (and for canisters the the CLI tool).
In this way, an attacker that somehow got access to your account could replace the keys, but not access what was inside the ledger, or your canisters for that matter. Then you could recover it with your seed phrase/security key regardless of what the attacker does.
The only limitation I see with this is that you can’t move existing Neurons, or non-ICP assets (ckBTC) into the hardware only account. But it’s a minor thing, e.g., an 8-yr neuron could be totally lost by an attacker who replaces your II, but with this method they can’t move the neuron and will be booted once you recover anyway.