Suppose I created a canister and want to stress test it calling it simultaneously from several threads.
Should the stress-tested be written in TypeScript and called as a regular Linux process on my PC or should stress-test be several instances of another (“second”) canister?
I think, better test from TypeScript on Linux, because unlike canister calls that my fail, TypeScript always works (if no errors and no memory overflow). Agree?
If you don’t care that the entire load comes from the same principal then creating a load canister that produces load for the stress test has advantages. You can create higher loads and it is easier to code. Failed calls should be rare and if they happen you can count them in the load canister to have an indicator.
I am doing some of my stress tests with https://github.com/infu/icblast and the blast notebook extension for VSCode https://marketplace.visualstudio.com/items?itemName=InternetBase.internet-base
Why are loads from WASM inside a canister are higher than from a multi-threaded Linux process?!
Because load from outside has to go through ingress messages which is subject to the consensus bottleneck of the subnet. So with that you are stress testing the subnet more than you are your canister. With inter-canister calls you don’t go through that bottleneck. So it’s easier to achieve higher loads.
Also signing 1000 requests per second isn’t easy on a Linux server either. But if you are ok with stress testing by anonymous calls then you can bypass that.
Update: I found out that the canister-to-canister queue size limit of 500 outstanding calls severely limits the inter-canister point-to-point tps that you can achieve. Your result will depend on whether the two canisters are on the same subnet or not.
Let’s say you send a batch of N calls with each heartbeat. It takes about 4 rounds (4 heartbeats) for the response to come back. That means that there are always 4N outstanding calls. With some variance there are usually between 3N-5N calls outstanding. Since the queue size is limited to 500 that means at N=100 you are already hitting the queue size limit from time to time. At a heartbeat rate of 0.7 per second that means 70 tps. In an experiment I reached a maximum of 75 tps (at N=125).
If the two canisters are on the same subnet then the round trip is faster than 4 rounds. Maybe you can actually achieve 4x the speed (I haven’t tried that yet). At that speed (4x75 = 300 tps) you would be already close to what you can achieve via ingress messages. So it seems that creating load for a stress test from another canister is still a viable and convenient option. You just have to make sure that the load canister is on the same subnet as the target.
Update to the previous post: I tried it out with a load canister on the same subnet as the target and achieved 267 tps (at N=500).