Can we somehow achieve or build a canister with a interface bahaving like a classic web server

Thanks for the info! I realise this behavior will be abstracted out eventually, but curious; is it possible for a developer to access a set of replica URLs from within a canister? I was under the impression that I could not, and looking at the PR it seems that they are being submitted via CLI rather than programmatically fetched in some way. Do you have any information about how that works in practice?

I am not 100% sure I know what you mean, but let me try to answer anyways:

Although not a really an intended use-case, it is indeed the case that a canister A can access the URLs exposed by canister B via the interface intended to be used by the HTTP Gateway, via an inter-canister call. Although my hope is that canisters will expose their features nicely via Candid-typed method calls, as there is no need to drop to the level of HTTP-encoded request between canisters.

I am not sure what you mean by CLI vs. programmatically fetched, though: A canister exposes the http_request query method, and then the HTTP Gateway will turn HTTP requests into IC query calls. There is no CLI in the strict sense involved, but also nothing is really “programmatically fetched”; the request is merely translated between the two protocols.

Is that how an asset canister knows to call its retrieve function when accessed by a url? by the creating of the http_request-method on the canister?

Does this mean we can fully customize our asset canisters as if they were regular canisters? Im curious how the asset canister compares to a regular canister at the protocall-level.

“Asset canisters” are just normal canisters whose primary purpose happens to be to store your frontend assets, but there is no special treatment on the protocol level.

3 Likes

Cool, thank you.
So where did the 2mb file limit on the asset canisters come from? (I know now there is no 2mb file limit with the new beta release)

My main question is will we have to use webpack for our frontend going forward? Or will we be able to just store the frontend-files-bytes on a canister and just serve the files to a browser with the http_request function?

Also is there docs out yet bout the http_request canister function?

Hi there - I was refraining from answering this question until I had signoff to make this post: Preview - Improved Asset Canisters. TLDR - we are supporting HTTP GET requests like a classic server now out-of-the-box!

The 2mb file limit came from a restriction on message sizes that came from wasm, however we’ve worked around it by supporting chunking in DFX.

Our improved canisters remove webpack as a dependency

We have some new documentation about the architecture here, with more to come once we settle on and promote 0.7.0 stable

8 Likes

Ah, sorry for the confusion. I was just referring to the CLI input from the Clap struct macro in the PR linked above, not for the actual release implementation. Just trying to understand the inner workings of ICP’s architecture as best as I can.

From what I can infer from what you said, a developer would, by design, not have direct access to the Replicas, only to the Canister that they are working within. The canister would simply forward the request to the replica, and funnel the response back into the canister.

I was mainly just wondering if it’s possible to interact with Replicas from the perspective of the developer in some theoretical case where you might need to run something on the “edge” of the network as Dfinity is doing here with the HTTP Gateway.

The 2mb file limit came from a restriction on message sizes that came from wasm, however we’ve worked around it by supporting chunking in DFX.

This is not entirely accurate: The 2MB restriction on message sizes is in “the replica”, i.e. the Internet Computer implementation. WebAssembly itself has no such restriction.

It seems that currently, the same limit applies to the size of update calls (user to canister, via consensus), query calls (user to canister, one node only) and inter-canister calls (canister to canister), both for the call and the response. One could imagine different limits (e.g. larger query responses), but it would at least break the uniformity of the messaging abstraction. In any case, there ought to be some limit, so this chunking in the HTTP Gateway is a great way to avoid issues here.

From what I can infer from what you said, a developer would, by design, not have direct access to the Replicas, only to the Canister that they are working within. The canister would simply forward the request to the replica, and funnel the response back into the canister.

Hmm, there might some terminology confusion here… are you confusing “canister” with “edge node/HTTP gateway”?

Developers create canisters (i.e. services). These run on a subnet, which is the same thing as saying that these run on many nodes/replicas simulatenously. Users can interact with the canisters; this interaction goes through the edge nodes (which route the request to a suitable node/replica that’s part of the right subnet for this canister).

I was mainly just wondering if it’s possible to interact with Replicas from the perspective of the developer in some theoretical case where you might need to run something on the “edge” of the network as Dfinity is doing here with the HTTP Gateway.

Ah, I see. Right now, user code (i.e. canisters) do not run on the edge nodes, but there is a vision that eventually, query calls (those that do not need to go through consensus and don’t change state) can be executed by the edge node directly, instead of by the node that takes part in a subnet. The Internet Computer is designed so that such an optimization can be introduced without the developers worrying about it, though.

4 Likes

:Gratitude for the clarification. I comprehend the need for the system message size limit, so now my question is with your statement here:

so this chunking in the HTTP Gateway is a great way to avoid issues here.

Where is the chunking happening in the specific? is it in the dfx tool? where the dfx tool uploads asset files in chunks to the asset canister?

Or is the chunking happening on the ic system level of some sort? if so can you laborate on where exactly the request is being chunked and how those chunks are sent? is it the “edge nodes” that are doing the chunking?

Does it work backwards too? if i need to respond to a http request with a large file will the system automatically chunk it for me? lets say the file is stored on my canister as bytes, if the whole file is 50mb, lets say a user requests the file with a public function update call, should the public function on the canister return with the whole mount of the bytes? or would i need to implement chunking on the canister?

Also if the chunking is on the edge nodes for the http_requests, does the chunking also work when communicating with the system through the standard ic api as is stated in the internet-computer-specification? or would the chunking only work when using the http edge node gateway?

The dfx SDK is able to chunk requests and send them to an asset canister. That canister is programmed to receive chunked files and to respond to http_request calls with chunked responses.

You could implement the same functionality in your own canister, but it won’t be present on all canisters, just the one that we build for you when you designate "type": "asset" for that canister in dfx.json.

Even better news - this code is open source! The code to upload chunked assets lives in ic-agent and the http_request interface lives in ic-utils/src/interfaces/http_request.rs.

3 Likes

The canister would have to implement chunking, but hopefully and eventually your CDK will provide a convenient high-level interface for that.

Yes, this chunking we are talking about is a feature of the upcoming HTTP gateway.

4 Likes

This is the best news thank you, good-portunity for the learn of the rust, i am looking for as close to metal as possible here thanks.

1 Like

In agent-rs, all requests to canisters implementing the HTTP interface are converted to query calls:

It seems like it should be trivial to inspect the request method and convert POST/PUT/DELETE/etc to an update call instead, but I’m assuming this code is only for development purposes and won’t affect the HTTP gateway on the network.

I took a look at dfn_http but I haven’t been able to figure out how/where that’s used or intended to be used.

I would contact the author but everything appears as dfinity-bot. It would be helpful if the commits showed the author as well as the committer.

@nomeata, do you know if update calls work with the HTTP interface for canisters?

Perhaps better for another thread, but I’d love some help understanding the process (in general) for proposing a change of this nature.

Is the ic repo a read-only public mirror, or does it accept pull requests?

How does the NNS fit in?

Thanks

I am facing a error when dfx deploy, do you know what cause the error.

ERROR in ic:canisters/supplier
Module build failed: UnhandledSchemeError: Reading from “ic:canisters/supplier” is not handled by plugins (Unhandled scheme).
Webpack supports “data:” and “file:” URIs by default.
You may need an additional plugin to handle “ic:” URIs.

The ic: URI is replaced with dfx-generated/. See the doc here: Add front-end assets :: Internet Computer

I think http_request only supports query method for now. We will need replica support to send update calls inside a query method.

@chenyan thanks Yan!

I would also like to propose that http_request can determine the HTTP status code as part of its return value.

I’ve already encountered a scenario where I need to return a 403 but am unable to do so.

Is there a better place to discuss this and make these proposals?

You can submit an issue at the agent-rs repo.

For update calls, @nomeata has a PoC here: GitHub - nomeata/ic-http-lambda: A HTTP-to-IC bridge (proof of concept)

I think these are both feature requests for the replica though.