Announcing the HTTP certification library for Rust

Developers Rust
1

Hello ICP builders :astronaut:

Nathan here from the Trust team :wave: Today I’m happy to announce the HTTP certification library for Rust!

Background

As a quick reminder, HTTP certification is a procedure that we follow in canisters to serve HTTP responses through query calls in a secure way.

Query calls are executed by a single replica node and do not go through consensus. This makes them fast, but insecure. Update calls are executed by all replica nodes on a subnet and go through consensus. This makes them slow, but secure.

Certification involves pre-calculating responses, putting them through consensus, and generating a certificate to prove that they have gone through consensus. These certificates are served alongside the pre-calculated responses, allowing clients to verify that the responding replica node has not maliciously altered the response.

HTTP certification is a more advanced version of this procedure that is used specifically for HTTP responses that will be received by HTTP clients, such as a browser. You can find out more about this in my previous post about response verification v2.

The library

The most common usage of HTTP certification today is through the beloved asset canister. To bring this functionality into their own canisters, developers previously had to copy/paste and maintain a lot of complex code from the asset canister. Now developers can leverage this code in a reusable way and build more flexibly using the HTTP certification library.

With this library, developers will be able to:

  • Serve certified assets from the same canister as their primary “backend” canister
  • Embed assets directly into a canister’s WASM, instead of uploading them at runtime
  • Create custom routing logic, such as:
    • Serving 404 pages in multi-page apps
    • Serving multiple frontends from the same canister
  • Certify more complex caching or streaming scenarios

As part of this release, we’ve published a user guide for the library along with example projects and guides demonstrating how to use the library to create an HTTP-compatible JSON API or serve static assets.

The library is available on crates.io, with the docs available on docs.rs and the source code available on github.

What’s next?

If the static asset example is too low-level, don’t worry. There’s another library in the works - the asset certification library will expose a higher-level interface purpose built for certifying and serving static assets in the simplest way possible. Stay tuned and you can expect to see the asset certification library in the coming months.

Feel free to reach out if you have any questions, suggestions, or issues with the library, I’ll be happy to help and I look forward to seeing the cool things that will be built with these libraries :rocket::new_moon:

28 Likes
2

Great work!

I believe the “single canister solution” (a canister that functions as a frontend server and a backend) is an important step towards microservices that are fully owned by the user.

It is convenient to have a single wasm. Users will be able to use a browser to create a canister, install some service wasm, manage and upgrade etc. without giving up control or ownership to any other party

5 Likes
3

Great work, can’t wait to give this a try! :heart_eyes:

3 Likes
4

@NathanosDev

Can we expect that these related crates will be maintained and what is the outlook for the Rust suite of certification crates?

ic_certified_assets
ic_certified_map
ic_certification

5

ic-certified-assets will likely be replaced at least partially by ic-asset-certification in the asset canister but I can’t speak for the exact plans that the SDK team has there.

ic-asset-certification is still in progress and will build on top of ic-http-certification.

ic-certified-map isa subset of ic-certification now and will be deprecated once some newer parts of ic-certified-map are merged into ic-certification.

ic-certification is used as the base for ic-http-certification so it will continue to be maintained.

3 Likes
6

Good to know

@Severin Could you perhaps give an update and outlook on ic-certified-assets crate and the ic-asset-certification module

7

ic-certified-assets is basically the asset canister that ships with dfx. The way it is structured does IMO not make it suited for any form of reusability. I plan to migrate it to ic-asset-certification as much as possible once that’s available. With that, I don’t think there would be much value to ic-certified-assets anymore, especially with how it is structured today (I think it doesn’t really allow reuse of anything useful).

While we have not discussed this in the team I honestly don’t really see a way forward to make ic-certified-assets a useful crate. What I could see is a crate that simplifies the use of the batch upload interface since one can use dfx to sync via that interface.

@Samer what do you think? Do you see a hole that ic-certified-assets could fill? Or what do you get as a consumer of it today?

3 Likes
8

Great work, @NathanosDev!
I have 2 questions: are there plans to bring this feature to other languages as well, and how much effort do you expect this to be?
Regarding serving frontend and backend from a single canister: I might be missing something here, but isn’t that already possible today (modulo the certification)? Here’s an example: GitHub - fxgst/azle-react: One-click dev env for developing canisters on ICP with JS/TS and a React frontend
Thanks!

1 Like
9

Yes, I’m especially interested the state_machine api that allows fast async chunked uploads with icx-asset. The api could also be consumed from browser.

Good to know where things stand. I have not made any decisions yet, but exploring the possibilities and available tools to create canisters that contain their frontend inside the wasm and serve it via certified query.

In Rust, we have the include_dir crate which works well for embedding assets in the wasm.

Every canister developer should be able to add a couple lines of code and the canister now implements the http_request and related endpoints and serves certified queries for an ‘onboard’ frontend.

static ASSET_DIR: Dir<'_> = include_dir!("src/frontend/build");

fn init() {
    init_frontend_assets!()
}
10

are there plans to bring this feature to other languages as well

There was BNT-13 for the community to bring some support for Motoko but I’m unsure of the current status.

For TypeScript/Azle, ideally, Azle would add support for using WASM modules so that we can use wasm-bindgen to generate a TypeScript version without needing to write it again from scratch.

In the longer term, supporting the WASM component model would make it much easier for libraries to be shared with other languages and I’m cautious about spending time writing libraries in other languages with the hope that we’ll have the component model sooner than later.

how much effort do you expect this to be?

The hardest thing about writing the library was trying to get the right interface design. There’s a lot of complexity that this library is trying to hide. I made multiple attempts before settling on the current design and I’m still not sure if I did it right yet :sweat_smile:

So if someone was to transpose the library into a new language and follow the same patterns then I think it would take significantly less time, but there’s still a lot to learn in terms of understanding what’s going on under the hood. I suspect that someone already familiar with the protocol could probably transpose the library in several weeks.

Regarding serving frontend and backend from a single canister: I might be missing something here, but isn’t that already possible today (modulo the certification)?

You’re right, the difference with the library is the certification. I’ll update the original post to say: “Serve certified assets from the same canister as their primary “backend” canister”.

Having said that, serving uncertified assets (or any uncertified query response from the IC for that matter) is extremely dangerous. It would allow a malicious replica to respond with anything it likes and expose a virtually unlimited number of attack vectors, depending on the canister being targeted. Disregarding certification reduces the security of your static asset serving down to the same level as a single replica subnet.

2 Likes
11

Every canister developer should be able to add a couple lines of code and the canister now implements the http_request and related endpoints and serves certified queries for an ‘onboard’ frontend.

I have the same vision for the ic-asset-certification library. The ic-http-certification library is a lower-level building block that allows for full flexibility, while ic-asset-certification aims to be much higher level and require as little involvement from the developer as possible.

1 Like
12

Greetings!

I would like to inquire about the current status of the ic-asset-certification library. From what I understand, it hasn’t been published on crates.io yet. Is it recommended and can it be reliably used in production environment?

Our idea is to use it in a dual-purpose canister to embed the user interface directly into the canister code, allowing interactions with the same canister. Currently, I’ve successfully experimented with examples from the repository:
Serving static assets source and Serving static assets over HTTP (custom) source
In terms of code clarity and simplicity, the first example is the absolute winner.

During my exploration, several questions arose:

  1. Regarding the ic-http-certification library published on crates.io: the latest listed version is 2.6.0, which matches the version in the current codebase. However, there are some discrepancies:
  • In the source code, HttpResponse specifies a <'a> lifecycle, which is absent in the published library.
  • The is no builder() for HttpResponse in published library
  • The published library does not export (or lacks) the add_v2_certificate_header function as well as add_certificate_header function referenced in the official guide: Serving static assets over HTTP (custom) guide
  1. Could you provide clarification on HTTP headers for HTTP responses? On one hand, in the source code, we see:
let mut headers = vec![
    ("strict-transport-security".to_string(), "max-age=31536000; includeSubDomains".to_string()),
    ("x-frame-options".to_string(), "DENY".to_string()),
    ("x-content-type-options".to_string(), "nosniff".to_string()),
    ("content-security-policy".to_string(), "default-src 'self';script-src 'self';connect-src 'self' http://localhost:* http://*.localhost:* https://icp0.io https://*.icp0.io https://icp-api.io;img-src 'self' data:;style-src * 'unsafe-inline';style-src-elem * 'unsafe-inline';font-src *;object-src 'none';base-uri 'self';frame-ancestors 'none';form-action 'self';upgrade-insecure-requests;".to_string()),
    ("referrer-policy".to_string(), "no-referrer".to_string()),
    ("permissions-policy".to_string(), "accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),layout-animations=(self),legacy-image-formats=(self),magnetometer=(),microphone=(),midi=(),oversized-images=(self),payment=(),picture-in-picture=(),publickey-credentials-get=(),speaker-selection=(),sync-xhr=(self),unoptimized-images=(self),unsized-media=(self),usb=(),screen-wake-lock=(),web-share=(),xr-spatial-tracking=()".to_string()),
    ("cross-origin-embedder-policy".to_string(), "require-corp".to_string()),
    ("cross-origin-opener-policy".to_string(), "same-origin".to_string()),
    ("access-control-allow-origin".to_string(), "*".to_string()),
];

On the other hand, the classic ic-certified-assets canister (which we are trying to avoid) declares only “content-type” and “etag” as mandatory headers(not mentioning “ic-certificate”) plus some conditional.
Meanwhile, recommendations such as those from this file: agent-js auth-demo differ as well, particularly in the “content-security-policy”.
We would like to understand which headers should be used to ensure maximum security and protection in the IC environment.

13

Is it recommended and can it be reliably used in production environment?

Until it’s released, I cannot guarantee there won’t be anymore breaking changes, but we have only one more planned breaking change before we will make the release. It will probably be merged tomorrow and the release made later this week.

Aside from the breaking change, there’s no reason not to use this library.

However, there are some discrepancies

The version in the codebase has not been updated since the last release. So there are many code changes that have been made in the meantime that are not present in the 2.6.0 version of the crate.

The version in the codebase will be updated to 3.0.0 when we do the release.

We would like to understand which headers should be used to ensure maximum security and protection in the IC environment.

There’s actually not anything specific to the IC here. This is just standard frontend web security. You can check this project for more information: OWASP Secure Headers Project | OWASP Foundation

14

Excellent! Thank you!