Agent-js can download chunks, but it’s the responsibility of the canister to determine when and how the chunks are returned. We don’t have a standard interface for the canister & agent to recognize that a payload is too large, and automatically retry with chunking, although that would be a nice feature.
In the meantime, the canister needs to offer chunks, or simply to expose the asset through a chunkable http query. In that case, you can fetch the asset using fetch or traditional http handling, and it doesn’t need to go through agent-js at all
Can you maybe record a video on this? I’m having a hard time figuring out what the different ways of retrieving data from a canister look like. So far I have this understanding regarding retrieving data that doesn’t fit into a single message, please correct me if I’m wrong.
Retrieving assets from an asset canister
To my understanding, when you request an asset on an asset canister via URL, this HTTP request reaches a boundary node first.
If accessed through ic0.app, a service worker will be installed in your browser. The service worker calls the HTTP interface of the canister directly issuing query calls to the http_request of the asset canister. The services worker then should handle streaming for you, unfortunately it’s not implemented yet. The boundary node just forwards this call to a canister, no transformation of the request is taking place.
If accessed through raw.ic0.app the boundary node translates the request into Candid which is then forwarded to the canister. If the asset is big enough to require streaming, the boundary node will take care of that for you. So you do not need to implement a streaming strategy in the Frontend, this happens automagically.
When using the agent-js, you’re basically in the same situation as with the service worker, no support for streaming out of the box yet.
So yeah, updates on Http Streaming! Http Streaming has been added to the serviceworker, and now works on ic0.app for large files by default. We also have a bounty for an AssetManagement interface in agent-js that is being worked on to make it easier to upload and download files from an asset canister