Yes, spot on. This is a known problem and I don’t think a complete solution is near. Essentially we have trust-on-first-use: As long as your first contact is uncompromised, later accesses work.
As you say, a proper solution likely needs a browser extension or a local proxy.