@ulan @domwoe thank you both. I agree even a small section would be great.
1 Like
@ulan Are there any plans to continue raising the DTS limit past 4X?
Now that canisters are able to hold up to 400GB, being able to run map reduce types of computations on that larger data will either require more rounds of consensus, or more complicated logic to chunk up the computations (more difficult developer experience).
Iād also imagine that the DeAI initiatives would be interested in having this limit raised.
4 Likes
@ulan Are there any plans to continue raising the DTS limit past 4X?
Yeah, we are planning to increase the limit to 8X-10X (40B - 50B instructions). Hopefully, it will happen in 1-2 months. Going beyond that is possible, but would require a strong use case because we are getting close to the hard limit of 500 DTS rounds.
6 Likes
I hope we can just keep pushing it as far as possible, we hit this limit on a regular basis and Iām pretty sure weāre not the only ones. For ICP to be a place of general-purpose computation we canāt have these kinds of limits.
This is one of the key limits I discuss with people as they contemplate deploying to ICP.
3 Likes
Does the hard limit come from checkpointing?
Really looking forward to this. The 221Bravo.app backend is constantly trying to get around this limit.
2 Likes
So the 4x was already implemented and is what enables Updates to have 20B instructions (thus the 10x is a 2.5x from there)?
Would Queries not run into the same issues because they do not reach consensus (implying DTS does not extend Query capacity but is also not needed for it)? Query instruction limits are constrained because they are free. Are there any known technical limitations of Query instruction limits?
Yes, message execution cannot be persisted across checkpoints (and a newly joining or catching up replica would need that in-progress state to start from). Meaning that at checkpoint heights all DTS executions have to be aborted and then restarted from scratch after the checkpoint.
Yes, the state that the query is executing on needs to be kept around until query execution completes. And although regular (not composite) queries only require the state of the respective canister, the implementation currently holds on to the full (subnet-wide) replicated state (which can easily be tens of GB of extra memory). And then there are composite queries, where you cannot know ahead of time which canisters are going to get called, if any, so you kind of need to hold on to the full replicated state.
1 Like
Thank you deeply for conveying the conceptual limitations of query calls. For context, I am interested in performaing AI inference on Chain and am hoping to establish a clear theoretical limit on WASM instructions per query so that I can build with an eye to the future. What I have been able to digest from your response:
- DTS is required for query calls because
- for a regular query the state of the canister @ time of the call is held in memory until the call is completed
- for a composite query the state of the entire subnet @ time of the call is held in memory until the call is completed
- The canister (entire state) must persist in memory not just the elements specific to the query call.
- The longer that the call takes, the more of an issue this could be for the entire subnet functionality by clogging ram.
- I could write an efficient canister intended only to do a single process process, a single specific query, which requires no excess RAM other than for the query process. However, there is concern that enabling the function in general would lead to general misuse and poor subnet performance.
- The limit using DTS would be the same as the upcoming limit for Updates of around 50 billion WASM instructions.
I appreciate any corrections and clarifications that you can provide.
I have heard that in the initialization of a canister that the limit is 200 billion WASM instructions. Iām curious how that works and if there is documentation as to why. Also someone mentioned that 500 billion WASM instructions might be some sort of absolute limit.
I suspect that the idea has already been brought up internally to enable the option preselect the canisters of interest for a composite query so as to avoid preserving the entire subnet state.
As a potentially simpler solution for the time being, can you rewrite your query APIs into update APIs? Theyāll be slower, but youāll be able to take advantage of DTS and use multiple rounds of consensus right away.
1 Like
You donāt need DTS for query calls. DTS stands for Deterministic Time Slicing. Queries donāt need determinism. Or time slicing.
I donāt work in Execution or Runtime, but AFAICT one could technically just bump the instruction limit for queries. Among the downsides of doing so would be having to hold on to the state for longer and taking up one of the (limied, not sure what the exact number is) of query execution threads for an extended time (so if you had enough long-running queries, it may happen that no new queries can be executed for a few seconds; for any canister). There are quite likely other reasons that I didnāt think about that would make it less than ideal to just bump the instruction limit for queries.
1 Like
@jeshliās original question, I am guessing, was not about composite queries. For a long-running non-composite query, theoretically, only a single replica needs to hold the state of the single canister that is being queried in memory for the time it takes to execute the query. If there was a way to pay for this, so as to avoid attacks, then the query could run arbitrarily long and regardless of checkpoints. If it runs too long then other replicas will no longer have the same state available and at that point the caller loses the possibility to query another replica just to compare the answers. There are many scenarios in which the caller has no desire to do that anyway.
There are ways to introduce payments for queries such as advance payments or query quotas.
For composite queries, say A is queried first by a user and then A queries B, the replica does not need to snapshot the whole subnet, nor does it need to know in advance to which canister the downstream query is directed (in this case B). The replica can snapshot Aās state when the query is made, and then snapshot Bās state sometime later when A makes the query call to B. It should not matter that the snapshots for A and B were not taken at the same time because we generally donāt guarantee timing for anything, not for queries nor for updates.
As said, only retaining the state of a single canister as opposed to the whole subnetās is not something thatās supported yet. I believe it is technically possible, but it would likely take time and effort to design and implement.
Also, IMHO one would have to significantly rate limit long-running queries because, as opposed to the average web server that can handle hundreds of concurrent requests, we impose a hard limit (4? 10?) on the number of concurrent queries doe to (I believe) each of them requiring its own canister sandbox process (whereas the average web server may not even need separate threads for each concurrent request it handles).
Regarding composite queries having a consistent snapshot of the subnet state, this is for better or worse the case now: if the user queries canister A, who queries canister B, who queries canister C, all three canistersā states will be from the same block height. I donāt thing thatās necessary; it may not even be desirable; but as with everything, applications may already rely on this property.
I will admit that this last one is a weak argument. But he first two (and who knows what others, as said, Iām just an interested observer, not a member of the Execution or Runtime teams) are annoyingly solid.
1 Like
Yes, understood that itās not supported yet. I was speculating about what can (or would) be technically possible because I think the original question was phrased āAre there any known technical limitationsā.
The limit on concurrent requests and the fact that they are expensive compared to traditional web servers could maybe be addressed by increasing the number of query-serving replicas in a subnet.
I hope that applications donāt rely on it. That would be a very weird property to rely on because if you run the same as a composite update call then you donāt have that guarantee.
Sure. But permanently increasing the number of query-only replicas would come at a significant cost increase. And would likely be inefficient. And (surprisingly enough) we donāt support query-only replicas, itās all just an idea at this point. And dynamically adjusting the number of replicas is yet another hugely complex feature away.
So yeah, itās all technically possible, but it will be expensive (both to canisters and in terms of development effort) and IMHO itās not by far the most pressing issue to solve (compare it with cheap integrated blob storage, or orders of magnitude more XNet bandwidth, as just a couple of examples).
2 Likes
Quick update on the DTS instruction limit.
I merged the change that increases the instruction limit for updates, timers, heartbeats to 40B (2x of the previous 20B limit): feat: Increase the instruction limit for update calls to 40B Ā· dfinity/ic@8e51868 Ā· GitHub
This should be included in the next replica release (to be deployed on the mainnet next week).
We are close to shipping the configurable Wasm memory limit feature (we were waiting for it in order to increase the limits).
6 Likes