Is this the case being described here when you talk about recursive self calls?
I meant both direct and indirect recursion: A → A and A → B → A.
From my understanding Option 4 allows A → B → A (with intermediate state being discarded), but does not allow B → direct loop to → B calls.
Option 4 would disallow both cases because A → B → A is an indirect recursion, so the question whether second A sees the changes of the first A applies to this case as well.
Would option 4 break any of these use cases?
Yes, Option 4 would break these cases. Option 1 would as long as these helpers do not communicate using the shared/global state, but rather take inputs, work on local state, and return outputs.
As a developer I don’t think I would make the assumption that I could destroy the query state because it “will be discarded anyways”. Most likely my code would be shared with the Update version and I would not assume the context that I was in. Moreover outside of say a DB system the idea that my state will roll back is not common. Much more likely I would assume that each part of the stack would act in a stateful way and that the overall behavior of the composite execution would act exactly as if it was an Update call except at the very end, after the execution has completed, where a query would have all the state changes tossed.
TL;DR IMO a composite query should execute exactly like an update until the very end where any state changes are discarded for the composite query. Anything else would be surprising.
It’s a tough call. At the first glance, I thought Option 1 is the most natural option, as each query call reverts its state. Then I realized this actually changes the semantics of await: await becomes a rollback point, instead of a commit point. If I want to change this function from composite_query to update, this function will suddenly have a very different behavior and it’s very hard to debug. In this sense, Option 2 is the least surprising one, as it emphasizes the composite aspect of the function, and preserves the semantics of await.
Also, are we allowed to call update method in a composite query, and what’s the expected semantics?
Most likely my code would be shared with the Update version and I would not assume the context that I was in.
Only synchronous code that doesn’t have await is likely to be shared between composite_query and update (that works the same for all options). Asynchronous code that calls self or other canister end-points would be difficult to share because composite_query cannot call update and vice versa.
Moreover outside of say a DB system the idea that my state will roll back is not common.
My main concern are IC developers that got used to working with the existing queries and rely on the rollback/don’t commit semantics.
await becomes a rollback point, instead of a commit point.
More precisely: await doesn’t rollback changes in the current query, but does rollback the changes for other queries.
Another way to look at it is: await doesn’t commit the changes to the global/canonical state from which other queries are executed.
I think your comment captures the essence of the difference between Option 1 and Option 2:
Option 1: changes are not committed at await points and at the end of each query.
Option 2: changes are committed at await points and at the end of each query except for the root query. At the end of the root query all committed changes in all canisters are rolled back.
If I want to change this function from composite_query to update , this function will suddenly have a very different behavior and it’s very hard to debug.
A similar argument applies to regular queries, right? If I want to change a query to update there will be some subtleties around state changes.
Also, are we allowed to call update method in a composite query, and what’s the expected semantics?
Instead of prohibiting composite queries from calling updates or having the developer have to worry about same vs xnet subnet calls, would it be bad design to transparently upgrade a call from a composite query to an update call automatically instead of having the call fail?
It’s quite difficult for dapp developers to have to deal with the intricacies of query vs update calls with the additional overhead of certified queries. Adding more complications like the above would just make it more difficult for dapp developers to use and test it.
There’s no sane way that I can imagine a dapp developer can test xnet composite query behaviour on their local replica and expect their mainnet behaviour to behave the same way
From a personal perspective, I would then rather choose a plain update call and have users deal with the latency than use a complicated and unpredictable cross canister calling mechanism like this.
Also, for the dapp I’m working on, our canisters are sharded to multiple subnets, so we would have ideally wanted to use this feature at some point in the future to reduce latency
This would break some canisters because queries have the contract of discarding state changes whereas updates commit state changes. For example, a query that doesn’t free allocated objects would be a valid query, but if we turn it into an update, then it would introduce a memory leak into the canister.
There’s no sane way that I can imagine a dapp developer can test xnet composite query behaviour on their local replica and expect their mainnet behaviour to behave the same way
Sorry about that. Supporting cross-subnet messages in non-replicated mode is a difficult technical challenge. It will take a while until we implement it, until then we are trying to ship a version that will be useful for some users.
I’m curious if we could get a version of the HTTPStreamingCallback that supports composite queries.
For example, let’s say I have a large file distributed across Canisters A and B; would it be possible to have the boundary node call a composite query that allows me to stream data from Canister A until it reaches a point that it needs to query Canister B for the next piece before relaying it back to the boundary node.
The trade-off is either to release composite queries now with potentially confusing rules or to work on replicated execution and release the proper query calls in 1-2 years.
I’d opt for the latter. Inserting a temporary “break” in the API (i.e. the model that the developer has to understand) puts the follow-up work on the critical path for the API to be unified again. Doing “the right thing” on the other hand is a plan that is more robust in face of priority changes, delays, etc.
I wonder, how many severe performance issues that we had in the past critically depend upon this feature? Do we have an estimate on what the overall cost of designing/implementing/testing/adopting this intermediate feature is compared to coming up with, e.g., canister design patterns that mitigate existing performance issues?
More precisely: await doesn’t rollback changes in the current query, but does rollback the changes for other queries.
I wonder if there is another dimension that can affect the state change. For example, the node answering the composite query is lagging behind the consensus. After the await, will the node catch up with the rest of the nodes? So even with option 1, the counter can be 0 and 1, just because of state catching up.
A similar argument applies to regular queries, right? If I want to change a query to update there will be some subtleties around state changes.
Correct, but adding the await difference would be yet another subtlety, besides the local state change. There is also this trick (not sure how widely it’s used) that people implement the same function as query and update calls. The frontend calls both methods, and use the result of query call to update the UI before the update commit happens. We lose this ability if we go with Option 1.
The results of the poll about the recursive query semantics in this thread and offline chats:
Option 1: nomeata, johan, dymayday, benji, Seb, icme, christian, dsarlis, ielashi, roman-kashitsyn, stefan-kaestle, ulan.
Option 2: claudio, jplevyak, chenyan, AdamS.
Option 3: cyberowl.
Other options: berestovskyy - make the behavior configurable by the caller.
I also chatted with several canister developers offline who did not express a preference here and viewed it as a theoretical question that is not relevant to their use case, where the “main” canister that calls multiple “helper” canisters and combines their results.
What’s the latest on composite queries? Has anything changed in dfx 0.14.1? I don’t see anything in the release notes but our tests for composite queries locally are showing some differences.
Any change you see in dfx is from us upgrading the included replica. You’d have to follow the replica changelog to see everything that changed there. All hashes of replicas included in a dfx release are in the releases
Looks like composite queries will be enabled on mainnet soon, would have happened already but the replica binary had some build reproducibility issues: Proposal: 123311 - IC Dashboard