Hey @Manu!
Thank you for posting this write-up, I am grateful that the community can participate in these conversations, we are learning a lot.
I have a few thoughts and questions on this specific part.
I think the above plan where canister-developers can migrate their canisters to a specific subnet of their choice, is great for subnets where a single entity controls who can install canisters on those subnets. If I am building canisters on the NNS/system subnets, Utopias, or subnet-rentals, then this feature sounds very useful.
However for public subnets where anyone can deploy canisters there are a few scenarios where problems might happen with that plan.
Scenario #1: If a few subnets get full at the same time due to a dapp’s or multiple dapps’ rapid growth, all other dapps on those subnets would quickly at the same time look for a low-load-subnet to migrate to. It’s likely that all other dapps would pick the same subnet (or the same few subnets) to migrate to since they will all be looking for the same low-load qualities/indicators. When that happens, the new subnets can get full just as fast since all other dapps are migrating to them at the same time. There is no way for dapp developers to coordinate with each other on which subnets to migrate to, to balance the load of the whole protocol optimally. This would cause dapp developers to frantically (since their customers are waiting) keep migrating their canisters to different subnets hoping others don’t pick the same subnets. Every dapp would probably end up having to try multiple different subnets to even things out. This would be very inefficient and might even cause more load across the whole network as many dapps are migrating all of their canisters to the same subnets multiple times all at once.
Scenario #2: If some bad actors want to DOS a specific dapp-service, the above plan makes it very easy for them to do that, and there is no way for the dapp-service to stop it. The bad actors can deploy a few thousand canisters running heartbeats to the same subnet as the dapp-service. When the dapp-service tries to migrate to another subnet (under the proposed plan), the bad actors can just follow them and migrate the bad-actor-canisters to the same subnet! And there is nothing the dapp-service would be able to do to stop it.