Payload Return Size

I know there is a soft cap on incoming message(about 2MB), but it appears that this extends to return values as well:

Status: rejected
Message: IC0504: Canister rrkah-fqaaa-aaaaa-aaaaq-cai violated contract: ic0.msg_reply_data_append: application payload size (4224046) cannot be larger than 3145728

I see that cancan has implemented retrieving chunks and assembling them on the client-side, but it would be nice if we could return large files via http_request with just one call. It makes src of media and image tags much easier. Iā€™ve also seen http_request_stream_callback floating around, so maybe this can be used for larger file sizes, but Iā€™m not sure how it works. Does anyone have any sample code they can point to?

4 Likes

Iā€™d like to know this too.

I planned on looking into this more, but havenā€™t yet: agent-rs/http_request.rs at b5094bdace81eaeb59ddfc5fbb27fe784e4d65ef Ā· dfinity/agent-rs Ā· GitHub

Where do you see that the incoming request payload limit is 2MB? Why is it a soft cap? Iā€™m curious about this too.

I believe this is because GC ( garbage collection) currently utilizes a 2GB user mode virtual address space for ā€˜32 bit computersā€™.
Purportedly they are going to be implementing a ā€˜trash compactorā€™ in the future to free up base memory.

Wait 2 GB or 2 MB???

Latest I could findā€¦.

And I think this is the updated thread that connects to the doc. Can we somehow achieve or build a canister with a interface bahaving like a classic web server - #39 by kpeacock

1 Like

The incoming message size for requests is like 3.x MB. It looks like intercanister it might be more, but that may just be for the replica. I havenā€™t tried sending more than 4MBs in a message in prod yet.

1 Like

Do you mind linking the code in the ic codebase that states this, if you have it handy? Or is this just from experience

Hereā€™s an example from icx-proxy in agent-rs that uses the streaming callback to return files of any size through a single HTTP request:

2 Likes

Interestingā€¦ seeing this created a bunch of questions / observations in my head.

  1. I believe this ā€œstreamingā€ solution is only for canisters that implement a http_request method, e.g. asset canisters. It wouldnā€™t work for canister calls on other methods.
  2. As OP mentioned, Iā€™m not sure how http_request_stream_callback, which is called by the code you linked above, works. Is this callback a function that runs in the Rust client or some arbitrary canister method running in IC? What is this callback token?
  3. How does it actually stream the response body (as Iā€™m guessing it doesnā€™t support streaming request bodies)? Is it basically HTTP/1.1-style chunked transfer encoding, i.e. break up the response body into chunks and call the callback with a chunk as an argument for each chunk? Thatā€™s crazy how itā€™s simulating HTTP/1.1 on top of Candid-encoded arguments and results wrapped inside CBOR-encoded request/response bodies all running on top of HTTP/2, which has its own streaming solution and binary encoding. Crazy stuff.
  4. Matter of fact, I donā€™t even understand what icx-proxy is. Is it a binary that we run somewhere and have our frontend client call via HTTP, which is then proxied to backend canisters? This way, our frontend client doesnā€™t need to call the canisters directly via Candid, CBOR, etc?
  5. The agent JS library apparently doesnā€™t support this. That makes this not so useful for, say, a mobile client that wants to stream a video file from an IC canister. I guess technically I could compile the agent Rust library to a wasm module and call that from JS? Seems like a headache.

Sorry, Iā€™m deep in the weeds tonightā€¦

3 Likes

Keep going into the weedsā€¦Iā€™d love to see some code that implements this stuff. Hoping to get there now that my ā€˜vacationā€™ is over. A week without a computerā€¦ughā€¦:disappointed:

3 Likes
  1. The asset canisters use a set of conventions, through their call interface, that support a form of streaming. The http_request method returns the first part of the response body (which may be the entirety of the response body), but if the response it too large to be returned in a single call, part of its return data includes a way to retrieve the rest.
  2. The callback runs in the (asset) canister, on the IC. Given a token, it returns the next piece of data in the stream, as well as a token for the piece after that.
  3. When using a streaming_strategy, icx_proxy sends the data piece by piece with hyper::Body::send_data(). I believe hyper in that case uses chunked encoding, but I am not certain.
  4. icx_proxy is a process that runs on the boundary nodes. It responds to HTTP requests by calling canister methods (http_request, and then possibly the streaming callback) and translating the results as an HTTP response.
  5. I donā€™t know if the intent is for the agent JS library to call http_request directly. However, the JS library should be able to make direct HTTP requests to retrieve data of any size, just as it can from any other web server.

I hope that helps. More generally, the way to use http_request and the streaming callback is:

  • Call http_request
  • If http_request returned a Callback streaming_strategy, call the callback, first with the token returned by http_request, then using the tokens returned by the callback, until the callback returns no token.

This is what icx-proxy does: agent-rs/main.rs at main Ā· dfinity/agent-rs Ā· GitHub

4 Likes

This is great! Iā€™ll try to work up an motoko example. Iā€™m guessing that I need to manually implement the returning of the callback, or will the canister do it automagically based on my motoko function return size?

Thanks, this was super helpful.

However, Iā€™m still not sure I understand what icx_proxy does.

When a browser makes an Ajax call to an IC canister to fetch data, what the agent library actually does under the hood is translate the canister RPC call to an HTTP request to /api/v2/canister/<effective_canister_id>/call and poll /read_state for the response (for update calls).

Is the proxy involved at all in this flow?

Or is it only involved when the URL path is anything but /api/v2/canister/<effective_canister_id>/{call,read_state,query} or /api/v2/status? In other words, if the path is something like /image/foo.png, then the proxy steps in to translate that into the appropriate HTTP request to /api/v2/canister/... with the image path as an argument in a call to a http_request canister method?

1 Like

Where do you see that the incoming request payload limit is 2MB? Why is it a soft cap? Iā€™m curious about this too.

Please take a look at this thread

Right, youā€™ll need to manually implement returning the callback.

If the agent is making those /api/ calls are going through HTTP, then yes, I believe that icx-proxy is involved.

The relevant code starts here: agent-rs/main.rs at main Ā· dfinity/agent-rs Ā· GitHub

icx-proxy forwards /api/ requests to the IC, and almost anything else to http_request on a canister. The exception is /_/, which in practice is forwarded in development to dfx in order to serve candid data.

1 Like

Oh, so icx-proxy runs in a local dfx start setup also then?

It is all starting to make sense.

1 Like

Have you already written mototko examples? Does it work? i also have this problem. thanks very much!!