Can we somehow achieve or build a canister with a interface bahaving like a classic web server

Developers
1

We’re facing huge struggle with the fact, that we cannot easily retrieve multiple assets for the web frontend. What we need is a canister, which is able to resolve arbitrary files by calling https://<canister_id>.ic0.app/path/to/asset.js or something comparable. Are there any plans or ideas on this topic?

I would consider this a fundamental requirement to build full fledged state of the art user interfaces on the IC within reasonable time frames. It looks like it’s only doable by bundling everything into one .js file.

I do not fully understand how the resolver currently works. In my understanding, the request to a specific canister should resolve somehow to its main function? Maybe someone can elaborate on this?

5 Likes
2

I actually don’t know the state of it, maybe we are not allowed to use it for now, but.

js-user-library/http_agent.js contains the next function (warning! I only have 0.5.5 installed locally, maybe it was changed):

retrieveAsset(canisterId, path) {
    const arg = IDL.encode([IDL.Text], [path]);
    return this.query(canisterId, { methodName: '__dfx_asset_path', arg }).then(response => {
        switch (response.status) {
            case "rejected" /* Rejected */:
                throw new Error(`An error happened while retrieving asset "${path}":\n` +
                    `  Status: ${response.status}\n` +
                    `  Message: ${response.reject_message}\n`);
            case "replied" /* Replied */:
                const [content] = IDL.decode([IDL.Text], response.reply.arg);
                return base64_js_1.toByteArray('' + content);
        }
    });
}

Which could be accessed by a user via <ActorObj>.__getAsset(path).

I never used it (maybe it doesn’t work), but at least it seems like the team is acknowledged about this requirement.

4 Likes
3

This also means that you should be able to do this from any client (even from CLI) - just call the __dfx_asset_path of your canister.

2 Likes
4

Oh. It looks like in 0.6.7 you need to create a separate canister for this.
At least dfx new's default hello-world canister does this. You should check it out.

5

Thanks! Yes I’m on the latest version. But the same holds true. There is no easy way to request particular assets without resolving them via JavaScript by utilizing the canister API.

6

You can retrieve files by importing the asset canister.

import asset from 'ic:canister/your_asset_canister';
const content = await asset.retrieve('file to retrieve');
const string = new TextDecoder().decode(new Uint8Array(content));
3 Likes
7

Yes, I’m aware of this. But this is not a solution in my eye. A modern web application will load the assets via sub/path/to/asset/whatever.file. Service worker for example cannot be register the proposed way. They need to be loaded via https.

2 Likes
8

I agree these are major problems. I’ve reached out to the team separately, hopefully we can get these capabilities added.

I don’t see why the static asset capabilities are being so narrowly defined right now. IMO webpack should have no part in the native functionality of the SDK. We need the ability to retrieve arbitrary static assets based on http paths, without the client having to know anything about ICP. This should all work with native simple http requests.

A number of capabilities are currently missing as I see it:

  • ES/JS modules are impossible to use natively (only possible to use them with a bundler. Bundlers are not using ES modules under the hood, they compile them away)
  • Service worker issues like you mentioned (I’ll take your word for it)
  • RSS feeds, won’t be able to just do an http grab of files off of ICP
  • Building any type of frontend that isn’t loaded from a JS file directly. What if I want to have an html file as my entrypoint? Or a txt file?
  • http2/3 multiplexing
14 Likes
9

+1000

Agree with all the points. Especially the bit about having webpack in the sdk.

Also can we have the sdk as an npm package please? Or at least a wrapper. Would make it easier to create something like NextJS but for dfinity :wink:

1 Like
10

Any news on this?

I’m trying to write an RSS server for a podcast feed. I need to return xml generated from the data in my canister. I’m new to webpack, but afaik I’ll never be able to return a simple xml file that can be parsed by a podcast app; it’ll always be surrounded by that initial loading screen and whatever bootstrap scaffolding html there is.

1 Like
11

I believe that the answer to this is that it is work in progress, about 80% complete. I can’t offer a release date and I’m not working on it myself so I cannot make a good projection for when it will be released but we do see this feature as highly desirable.

5 Likes
12

Dudes, you just caused a 100 message thread internally and a demo. Hopefully the author of the demo can post some details.

3 Likes
13

Guess now I have to…

Yesterday I created a small proof of concept of how something like this could look like to developers. This is in no way official, so treat this as built by some rando in the community…

The very small program at https://github.com/nomeata/ic-http-lambda/ implements an HTTP-to-IC bridge, encapsulating HTTP requests as Candid data, and sending it to the canister id mentioned in the hostname. This means you can go to https://jwksz-eqaaa-aaaab-aaaaq-cai.ic.nomeata.de/ and the full response (HTML, content-type, cookie headers) are under control of the canister.

As a demo, I wrote a canister that implements a simple Telegram bot, you can play with it at http://t.me/InternetComputerBot and see the code at https://github.com/nomeata/ic-telegram-bot.

Currently, I am hosting this service myself, outside of the IC infrastructure (actually on Amazon Lambda…), but it would be feasible (but not necessary, as I have just proven) that a service similar to this could eventually be provided as part of the Internet Computer platform, with a more official domain, maybe with a name registration service…

I agree that this feature is great to onboard developers, but it also its its problems: E.g. you can initiate HTTP requests from the canister. And worse, you throw out more or less all the amazing security guarantees of the Internet Computer (e.g. responses are tamperproof, user identities are verified by the system). So yes, things are moving … but let’s get the platform properly live first!

18 Likes
14

This is awesome, thank you nomeata!

I was in the process of building something similar so I can turn my canister’s data into a parseable rss feed.

Are we invited to use https://<canister_id>.ic.nomeata.de/? It would be handy as I build out and test my app, while we wait for a more formal/secure solution.

2 Likes
15

@nomeata when I run ic-http-lambda locally, the response tells me Use https://<cid>ic.nomeata.de/!. That won’t work for locally-running canisters, right? What’s the url when running locally?

I assume the address is supposed to be https://<cid>.ic.nomeata.de/, with a ‘.’ between <cid> and ic, but of course neither work for me and my local canister.

I’ve tried <cid>.localhost:7878, <cid>.ic.localhost:7878 and <cid>ic.localhost:7878 (7878 is the port spit out when I run cargo run from the ic-http-lambda folder). All of those tell me to use the ic.nomeata.de address.

1 Like
16

Sure, you can use that for experiments, but don’t complain if it breaks :slight_smile:

For local use, you have to

  • Cange let url = "https://gw.dfinity.network"; (if you also want to use a local replica)
  • Change the line .and_then(|h| h.strip_suffix(".ic.nomeata.de").map(|x| x.to_owned())) to your domain (which for local development p
  • or, if you test just one caniser, just hardcode
    let cid = ic_types::Principal::from_text("your-canister-id".to_string()).unwrap()

PRs against https://github.com/nomeata/ic-http-lambda to make local use more convenient (e.g. a flag --always-canister <cid>, a flag --endpoint <url>…) are welcome.

3 Likes
17

I submitted a PR that adds local replica support. I’ve been using it and it’s working for my needs.

I also was able to convert your ic-telegram-bot’s requst/response code into motoko so I can write my server in motoko. Works great, but took some finagling as I tried to wrap my head around the Rust code. Maybe someone can save some time by checking out my code, and maybe someone has some input as to how it can be improved.

Again, thanks @nomeata for your work on this! You’ve made it possible for me to begin work on my podcast host.

4 Likes
18

Cool stuff!

Instead of body: [Nat8]; you can use body : Blob; they have compatible types in Candid world, but Blob is more efficient in Motoko (but then, we still don’t have all the operations on Blob that you might expect, so maybe array is better for now.)

19

@nomeata I’ve been working with the ic-http-bridge locally and it’s working great. I did have to increase the timeout in the waiter because update calls take longer than 5 seconds to complete. Unfortunately all my requests have to be upgraded to update requests even if they don’t modify state (see this thread)

Because I need to increase the timeout, I can’t use your implementation at ic.nomeata.de. I just get timeouts if I try that, since 5s is not enough. So I’m trying to release the modified ic-http-lambda code as my own function on aws lambda. I’ve done a little with aws, but never a lambda function and am getting a little tripped up trying to deploy it.

When setting up the lambda function, I don’t see Rust as an option when choosing the runtime language. I do see “Provide your own bootstrap on Amazon Linux 2”, which I assume is what you must have done to get this to run.

I noticed this runtime. Is that what you used? If so, did you use the AWS CLI or the Serverless framework?

I also noticed this in the ic-http-lambda readme: “To build for Amazon lambda, and hence musl, this is using a patched agent-rs without an openssl dependency for now.”

Will I need to make changes to (patch?) the runtime code (agent-rs?) to get it to work the way you did?

Sorry if these questions seem vague, but that’s that state I’m at right now in getting this to work. I would greatly appreciate your help in getting my own version of ic-http-lambda up and running!

20

At Fleek we’re trying to build a solution that allows you to surface frontend canisters like any other web app. You can have basic CRA, add these files: dfx.json · GitHub and simply dfx deploy --network ic. After it’s deployed you can access website on URL like https://qajud-uiaaa-aaaab-aac6a-cai.ic.fleek.co.

Here’s how it works behind: Initial request hits our server, that returns simple bootstrap script and installs a service worker to your browser. After service worker is ready, canister assets are fetched directly from IC gateway (thanks to SW there’s no proxy between your browser and IC gateway). Also it’s “bot-friendly” (when SW is not available, like search engines, link previews, …), these kind of requests are proxied to IC.

image (14)
image (14)798×452 54.1 KB

9 Likes