Proposal: Composite Queries

Is this the case being described here when you talk about recursive self calls?

I meant both direct and indirect recursion: A → A and A → B → A.

From my understanding Option 4 allows A → B → A (with intermediate state being discarded), but does not allow B → direct loop to → B calls.

Option 4 would disallow both cases because A → B → A is an indirect recursion, so the question whether second A sees the changes of the first A applies to this case as well.

Would option 4 break any of these use cases?

Yes, Option 4 would break these cases. Option 1 would as long as these helpers do not communicate using the shared/global state, but rather take inputs, work on local state, and return outputs.

As a developer I don’t think I would make the assumption that I could destroy the query state because it “will be discarded anyways”. Most likely my code would be shared with the Update version and I would not assume the context that I was in. Moreover outside of say a DB system the idea that my state will roll back is not common. Much more likely I would assume that each part of the stack would act in a stateful way and that the overall behavior of the composite execution would act exactly as if it was an Update call except at the very end, after the execution has completed, where a query would have all the state changes tossed.

1 Like

TL;DR IMO a composite query should execute exactly like an update until the very end where any state changes are discarded for the composite query. Anything else would be surprising.

1 Like

It’s a tough call. At the first glance, I thought Option 1 is the most natural option, as each query call reverts its state. Then I realized this actually changes the semantics of await: await becomes a rollback point, instead of a commit point. If I want to change this function from composite_query to update, this function will suddenly have a very different behavior and it’s very hard to debug. In this sense, Option 2 is the least surprising one, as it emphasizes the composite aspect of the function, and preserves the semantics of await.

Also, are we allowed to call update method in a composite query, and what’s the expected semantics?

1 Like

Thanks for the insightful comments @jplevyak and @chenyan!

Most likely my code would be shared with the Update version and I would not assume the context that I was in.

Only synchronous code that doesn’t have await is likely to be shared between composite_query and update (that works the same for all options). Asynchronous code that calls self or other canister end-points would be difficult to share because composite_query cannot call update and vice versa.

Moreover outside of say a DB system the idea that my state will roll back is not common.

My main concern are IC developers that got used to working with the existing queries and rely on the rollback/don’t commit semantics.

await becomes a rollback point, instead of a commit point.

More precisely: await doesn’t rollback changes in the current query, but does rollback the changes for other queries.

Another way to look at it is: await doesn’t commit the changes to the global/canonical state from which other queries are executed.

I think your comment captures the essence of the difference between Option 1 and Option 2:

  • Option 1: changes are not committed at await points and at the end of each query.
  • Option 2: changes are committed at await points and at the end of each query except for the root query. At the end of the root query all committed changes in all canisters are rolled back.

If I want to change this function from composite_query to update , this function will suddenly have a very different behavior and it’s very hard to debug.

A similar argument applies to regular queries, right? If I want to change a query to update there will be some subtleties around state changes.

Also, are we allowed to call update method in a composite query, and what’s the expected semantics?

A composite query cannot call an update.

1 Like