Are you sure? I just tested a bit and I could not reproduce this. I suspect you forgot/did not know that you don’t only have to edit (assuming the project dfx new hello) src/hello_frontend/assets/.ic-assets.json5 but also src/hello_frontend/src/.ic-assets.json5.
Yes this should work. In my own testing, I needed to do a hard refresh though. The browser cached the allow_raw_access: false setting
Did you edit both .ic-assets.json5 files?