Can we somehow achieve or build a canister with a interface bahaving like a classic web server

Developers
21

Because I need to increase the timeout, I can’t use your implementation at ic.nomeata.de.

Which timeout do you need? I can change my setup if you want. But maybe better to let you host your own :slight_smile:

I do see “Provide your own bootstrap on Amazon Linux 2”, which I assume is what you must have done to get this to run.

I think so, yes. It says “Custom runtime on Amazon Linux 2” here. (Not an AWS expert myself.)

Will I need to make changes to (patch?) the runtime code (agent-rs?) to get it to work the way you did?

Nope, if you look at my Cargo.toml you’ll see that it pins the patched version of the agent:

[patch.crates-io]
ic-agent = { git = "https://github.com/nomeata/agent-rs", branch = "joachim/musl-hacks" }
ic-types = { git = "https://github.com/nomeata/agent-rs", branch = "joachim/musl-hacks" }
22

@nomeata I tried putting together a PR for the increased timeout, but I can’t get ic-http-bridge to run locally anymore, even without the change. When I try to make a request to http://127.0.0.1:7878/, it causes the local dfx server to crash, which then restarts with a new port number, and ic-http-bridge respond with Could not reach the server presumably because it’s looking on the old port. Relaunching ic-http-bridge with dfx’s new port just repeats the cycle and I never get a response.

This is the crash reported by the process started with dfx start:
thread 'Http Handler' panicked at 'Opening old round db failed 493301', src/cow_state/slot_mgr.rs:469:37

23

In any case, when I was able to get this running, the timeout I needed to increase was on line 164:

let result = if result.upgrade {
    // Re-do the request as an update call
    agent.fetch_root_key().await?;
    let waiter = delay::Delay::builder()
        .throttle(std::time::Duration::from_millis(500))
        .timeout(std::time::Duration::from_secs(90))    // <-- increased from 5 to 90
        .build();

90 seconds is definitely more than I need; I was just using that for testing. I recall responses coming back in around 30-40 seconds.

1 Like
24

EDIT: did you try using the dfx start --clean command yet?

1 Like
25

Which command? I’m not seeing --clean as a flag on any of the dfx commands. In the past, I’ve cleaned by deleting the .dfx folder, but doing that and then rebuilding results in the same cid for my canisters, so there must be something else that’s not getting cleaned up.

edit: Not sure how I missed dfx start ~~clean but things are working now
edit edit: those tildes are supposed to be dashes, but editing a post on this forum causes a 403 error if it includes dashes…

1 Like
26

Can we implement Telegram bot all by canisters in the future?

27

Sure, why not? (Assuming you use a HTTP bridge like the one I built, or eventually building on something similar that makes it into the offical offering.)

3 Likes
28

Hey, I’m new here (and newbie programmer overall), came from Python that I fell in love, now Dfinity is my new romance, but I need to make a big jump :D.

I like writing telegram bots in python, so doing one on IC seems to be perfect example to start with.

Questions:

  • Whole bot logic is pure rust, and I can change in without problems?
  • Bot is connected to ic-http-lambda running on AMAZON LAMBDA service? Is this like webhook and bridge between old rusty internet and new shiny IC? : D
  • Live demo is pointing to your local environment? Or it’s live on IC and cost cycles?
  • Can I write something to fetch data from outside http APIs and save it to IC, or forward to telegram chat?

Sorry for stupid question if asked, but the whole thing is still shaping in my head :slight_smile:
Thanks for help, and great job!

1 Like
29

The ic-http-lambda is just a side proof of concept of mine, to show what’s possible with an HTTP interface. The fact that I run it on AWS is immaterial, once a feature like that becomes official nobody will remember my little experiment :slight_smile:

Can I write something to fetch data from outside http APIs and save it to IC, or forward to telegram chat?

No, there is no good story for doing HTTP requests from a canister; it is somewhat at odds with replicated deterministic execution. So not for now.

1 Like
30

Judging from feat: add an http-request interface and icx-http-server bin by hansl · Pull Request #120 · dfinity/agent-rs · GitHub it seems that an “offical” version of this feature is in the making, so no need to worry about my little AWS experiment :slight_smile:
(That one not yet with upgrade-to-update call, though).

5 Likes
31

Interesting. If I’m interpreting this PR correctly, it looks like they are experimenting with sending HTTP requests via a set of Replica URLs by creating an intermediary canister that forwards the response back to a parent canister to circumvent wasm’s lack of host knowledge. I’m still quite new to ICP, so might be missing something in that pipeline (or misinterpreting it, also pretty new to Rust!.. and wasm :stuck_out_tongue:)

Wondering if we have any information on when (or if) this feature might be incorporated into the IC CDK? This will inform whether I wait for a bit so I can do it on the IC, orrrrrrr make a bridging 3rd party server to forward responses into my main backend canister.

2 Likes
32

Close: it’s not an intermediate canister, but rather a gateway service that is built into the “edge” of the internet computer - but as a developer, this is not something you need to worry about. The effect is simply that your canister is directly accessible from any HTTP client in the world: browsers of course, but also services invoking web hooks etc.

Docs should be coming soon™

5 Likes
33

Thanks for the info! I realise this behavior will be abstracted out eventually, but curious; is it possible for a developer to access a set of replica URLs from within a canister? I was under the impression that I could not, and looking at the PR it seems that they are being submitted via CLI rather than programmatically fetched in some way. Do you have any information about how that works in practice?

35

I am not 100% sure I know what you mean, but let me try to answer anyways:

Although not a really an intended use-case, it is indeed the case that a canister A can access the URLs exposed by canister B via the interface intended to be used by the HTTP Gateway, via an inter-canister call. Although my hope is that canisters will expose their features nicely via Candid-typed method calls, as there is no need to drop to the level of HTTP-encoded request between canisters.

I am not sure what you mean by CLI vs. programmatically fetched, though: A canister exposes the http_request query method, and then the HTTP Gateway will turn HTTP requests into IC query calls. There is no CLI in the strict sense involved, but also nothing is really “programmatically fetched”; the request is merely translated between the two protocols.

36

Is that how an asset canister knows to call its retrieve function when accessed by a url? by the creating of the http_request-method on the canister?

Does this mean we can fully customize our asset canisters as if they were regular canisters? Im curious how the asset canister compares to a regular canister at the protocall-level.

37

“Asset canisters” are just normal canisters whose primary purpose happens to be to store your frontend assets, but there is no special treatment on the protocol level.

3 Likes
38

Cool, thank you.
So where did the 2mb file limit on the asset canisters come from? (I know now there is no 2mb file limit with the new beta release)

My main question is will we have to use webpack for our frontend going forward? Or will we be able to just store the frontend-files-bytes on a canister and just serve the files to a browser with the http_request function?

Also is there docs out yet bout the http_request canister function?

39

Hi there - I was refraining from answering this question until I had signoff to make this post: Preview - Improved Asset Canisters. TLDR - we are supporting HTTP GET requests like a classic server now out-of-the-box!

The 2mb file limit came from a restriction on message sizes that came from wasm, however we’ve worked around it by supporting chunking in DFX.

Our improved canisters remove webpack as a dependency

We have some new documentation about the architecture here, with more to come once we settle on and promote 0.7.0 stable

8 Likes
40

Ah, sorry for the confusion. I was just referring to the CLI input from the Clap struct macro in the PR linked above, not for the actual release implementation. Just trying to understand the inner workings of ICP’s architecture as best as I can.

From what I can infer from what you said, a developer would, by design, not have direct access to the Replicas, only to the Canister that they are working within. The canister would simply forward the request to the replica, and funnel the response back into the canister.

I was mainly just wondering if it’s possible to interact with Replicas from the perspective of the developer in some theoretical case where you might need to run something on the “edge” of the network as Dfinity is doing here with the HTTP Gateway.

41

The 2mb file limit came from a restriction on message sizes that came from wasm, however we’ve worked around it by supporting chunking in DFX.

This is not entirely accurate: The 2MB restriction on message sizes is in “the replica”, i.e. the Internet Computer implementation. WebAssembly itself has no such restriction.

It seems that currently, the same limit applies to the size of update calls (user to canister, via consensus), query calls (user to canister, one node only) and inter-canister calls (canister to canister), both for the call and the response. One could imagine different limits (e.g. larger query responses), but it would at least break the uniformity of the messaging abstraction. In any case, there ought to be some limit, so this chunking in the HTTP Gateway is a great way to avoid issues here.

From what I can infer from what you said, a developer would, by design, not have direct access to the Replicas, only to the Canister that they are working within. The canister would simply forward the request to the replica, and funnel the response back into the canister.

Hmm, there might some terminology confusion here… are you confusing “canister” with “edge node/HTTP gateway”?

Developers create canisters (i.e. services). These run on a subnet, which is the same thing as saying that these run on many nodes/replicas simulatenously. Users can interact with the canisters; this interaction goes through the edge nodes (which route the request to a suitable node/replica that’s part of the right subnet for this canister).

I was mainly just wondering if it’s possible to interact with Replicas from the perspective of the developer in some theoretical case where you might need to run something on the “edge” of the network as Dfinity is doing here with the HTTP Gateway.

Ah, I see. Right now, user code (i.e. canisters) do not run on the edge nodes, but there is a vision that eventually, query calls (those that do not need to go through consensus and don’t change state) can be executed by the edge node directly, instead of by the node that takes part in a subnet. The Internet Computer is designed so that such an optimization can be introduced without the developers worrying about it, though.

4 Likes