Certified Assets from Motoko (PoC/Tutorial)

Every now and then someone asks whether Motoko canisters can use certified variables, or whether they can serve HTTP requests with certification, and I always responded that yes, Motoko can do that, all that’s missing are a few libraries. But I figured I should put my hand where my mouth is, and actually demonstrate that it’s possible.

So here we go. If you go to https://ce7vw-haaaa-aaaai-aanva-cai.ic0.app/ you will see that

To prove the latter claim, here is the commented code; you can also browse the full repository.

Actually, the code is not here, because of this annoying 403 bug of the forum where after carefully editing and formatting the post, it says I can’t post it. Too many URLs maybe? :frowning: So unfortunately, you’ll have to go to https://gist.github.com/nomeata/f325fcd2a6692df06e38adedf9ca1877 to read it.

10 Likes

Oh, even nicer: This file can be loaded into Motoko Playground! So if you go to Motoko Playground - DFINITY you can click Deploy, and see it in action! You’ll have to take the canister it reported by Motoko Playground and construct the .ic0.app URL from it. And certification will fail until you send a message (most easily done directly from Motoko Playground) or upgrade the canister.

6 Likes

Great works.

This demo is very helpful to understand certificate.

I have some questions:

  1. If leave_message is not called, the signature of the certificate should be the same, but each time the certificate obtained has a small part of the string different, and the guess /time field is different, is it right?
  2. The /time field is not included in the signed message, can the node theoretically modify /time field?
  3. Similar to “http_assets” in asset_tree, is there any other? What are the other certificate application scenarios, and how do they need to construct this asset_tree.
1 Like

Correct

No: while it is possible to omit data from a Merkle tree without breaking the signature, you can’t add or modify fields.

But since the spec says that the time field ought to be there, a validating client should reject a certificate that doesn’t include it.

The canister signature schemes used by the Internet Identity is another protocol based on certified variables. I don’t know of any other common protocols yet, but assume that there could be application-specific ones.

1 Like

Thanks for this guide.

I’m trying to wrap my head around certified variables, and it seems like there are two Merkle trees (and their respective root hashes) at play here.

  1. The system-level Merkle tree representing the IC state tree. This is returned by CertifiedData.getCertificate in Motoko, which can be passed to a client in the response of a query call for verification. It is always passed to a client who polls for the response of an update call. This Merkle tree includes the 32-byte blob that represents the certified variable at /canisters/<canister_id>/certified_data. This Merkle tree is updated and appropriately pruned by the replica.
  1. The application-level Merkle tree representing the application-specific data that needs to be certified and thus made secure. This is constructed and maintained in memory inside the canister, and must be updated and pruned by the canister logic. The root hash of this Merkle tree is stored as a certified variable inside /canisters/<canister_id>/certified_data using CertifiedData.set in Motoko, and is signed by the subnet by virtue of being part of the IC state tree, whose root hash is always signed by the subnet.

It seems to me that both Merkle trees are necessary to implement certified variables—having one is not enough. The second Merkle tree is directly related to the first Merkle tree, because the root hash of the second tree is stored inside the first tree (whose root hash is returned to the user as part of a Certificate).

The client that queries for a certified asset (or other certified data) must validate both Merkle trees by recomputing the root hash and comparing it with the provided root hash, for each tree. The client only needs to validate one signature, however, and that’s the signature of the root hash of the first Merkle tree for the IC state tree.

Does this sound right to you @nomeata? This was not trivial to figure out.

1 Like

Yes, absolutely! Nice summary :slight_smile:

Note that always pruned merkle trees are passed around, revealing different information. Certified.getCertificate reveals the certified_data of the canister, while the partial merkle tree polled in an update call reveals the request status. But they are both part of the same per-subnet state tree.

1 Like

Any way to do this from local?

I commented out the update_asset_hash and was still able to get to the page without raw

@Iceypee you’re always using raw when you’re running DFX locally.

1 Like

Is there a memory limit/cycles limit to this library or can I just do a debug_show and aggregate every single data structure into a single call to this function from the example code (specific part shown below) as a really lazy way of doing things. Basically making storage O(2n)

Also I am curious about cost to store, and read and write costs to it as well in terms of cycles if someone knows.

Everything on the IC has memory and cycles limits. I would assume that this library does not use stable memory, therefore you’d have the standard 4GB WASM heap size limit, and you always will have the default cycles limit

1 Like

Just putting a reminder for myself here if you want to make this work with local have to use

http://<canister-id>.localhost.8080 inside the browser …replace the localhost.8080 with whatever your local networks.json has it saved as

note the http:// which isnt listed in the image i posted

Hey, can you lend me some good example of how you created merkle tree in :
func asset_tree() : HashTree {}, like for canister holding different assets.

The OP created a new library thats for that

The src folder has the data structures, the demo has a decent demo that makes sense after a while

1 Like

Just curious, if we aren’t using this for web but just verifying a variable was certified how do we use this?
Like I get to the point I get a certificate like below. Just don’t know what to do next

“ic-certificate”;
“certificate=:2dn3o2R0cmVlgwGDAYMBgwJIY2FuaXN0ZXKDAYIEWCChTvD6h3hfSgcjeRKzM3XFobHLpnlXps464aXyTcFAjIMBggRYIMn7Fh/Jpl1ktN3cqxgazvx9IVZ892DSU3XG0wEhlfv3gwGDAYIEWCC30EUaadp/RzFjnF18VQy0bUkTdv96ZjkyEknttzTVmIMBggRYICpaJeksf3PZheDaCQQJvTr3EctaE2EssvpEwCrzUDHtgwGCBFggsx2qrIfHQM3n43/LeUYAAjOMODKeOJWNZU4PhJ1fnmmDAYMBgwGDAYIEWCAmtfAXEZJ4EIlehm5wfMZwNcpSyguZDx6G2idQwrM3CIMBggRYILljkWLcF5Ya/pU4AboT6l1m0raXp0rWaLZO5IO8sKWFgwJKAAAAAAFwEBEBAYMBgwGDAYMCTmNlcnRpZmllZF9kYXRhggNYIK7zGuEPyWiedHYW6goByZjgrw4+ZY8BGnITmzlO77H4ggRYIIPFa/FN3j0o3mbGkrX8nZfp3ZhbatcrD+b4ToqN89yyggRYINu5ywdu+JvZYP7+Ki1+TiqO5eVNBc14ujXSRmCK13FJggRYICaeE8DhtCpBM1RGT7YRgQ84K7ZDKRljZNjtF/ldJKK1ggRYIN5P+N5JDXuhgxCZMTcsA0WusBUb+uUgGPYiloxpmYLXggRYIFESNBOtRoQEc1weMAzbZ1bSfXL315p1xzHRQR5ZJQyVggRYIBQe/0rz2S4WXMfozNz1uOYhXT7RZOteXvz9ut0dKDrHggRYIP9XcgQ7xDmhW7YUfdXCgvEFQnhjIv/+uivs6OgcQMenggRYINUqe/Me7Ot/8BYFxFvhKKE2SbcugHND6fSimK5yn362ggRYIEIR248FAGjtk5hmjUGGtsEw1b2t6DScYUhfRZxzjRZBgwGCBFggzFeBCfxdveWRz3nLP6YybNzN2fqI6idZLKiIgkNmr+ODAkR0aW1lggNJnpTRzbe067oXaXNpZ25hdHVyZVgwmAyYS5hWoqHwyYo/WnOk1tHd/h4fV+ZOSaprcISpiqTkZ4zefJ300xURJzyzEjEaamRlbGVnYXRpb26iaXN1Ym5ldF9pZFgdWBtaovEMXyIeRd50os7K/T2gdr452wkHy/VUuwJrY2VydGlmaWNhdGVZAlfZ2feiZHRyZWWDAYIEWCC56xOj0DzPg24+oG8pSiSAdafQoBPNgiEF9+h7inQ+74MBgwJGc3VibmV0gwGDAYIEWCBTY9mSHEaKwaDAR1EkoXA7Duq0TxKv5LtyGgWeq/vM3oMBgwGDAYMBgwJYHVgbWqLxDF8iHkXedKLOyv09oHa+OdsJB8v1VLsCgwGDAk9jYW5pc3Rlcl9yYW5nZXOCA1gb2dn3gYJKAAAAAAFwAAABAUoAAAAAAX///wEBgwJKcHVibGljX2tleYIDWIUwgYIwHQYNKwYBBAGC3HwFAwECAQYMKwYBBAGC3HwFAwIBA2EArSiiv6rmySSO8eo+T0vG6ikQHOUzsAdTSbuGqIU1whbOQ0yaoBi4dGVEHdmX7SMnDish4SbHTxYpO9uikDPoWOzVC9p74baSpAHhVOuHtl3PCYNjXleOPqL22bAsfHncggRYII0bb/PggEBVd5jVU9ZK+ViZ8loBZuOxJ20cjdvfWrcFggRYIF0tHJQ49uHbod+twsUPUYtjGCYo081ptJRNWMs2qpicggRYIJw4PrjSp8hsakQTVQaijycHL+Lb7z6e6a9HHs3AY7PTggRYIGoX9qb27SEZm4g9MxiDSItkMnfhcHLoLRzEA2u1LKNmggRYIN8RJENd8cm64fE0TvP9pqYPj699BnIONfATSdimT8lkgwJEdGltZYIDSaat14jlheq6F2lzaWduYXR1cmVYMKtCOw0M+/FEjsIfdMKbrc8HyCkPVmu8da+5BUOHN9Ta/UaPiDmQkKRDvKt+iFQZuw==:, tree=:2dn3gwGDAktodHRwX2Fzc2V0c4MBggRYIIaF9LgbVX9sUOfQiqKFJCoaGe1I/yaNWsmeoEYmWfj+gwGDAYIEWCCZVnRMhVE/k2cWMYAg4mzI4fspFWvo77a+d4Sy84upBYMCSS9nZXQvamtqa4IDWCAH556XlJn1n9w6t8kWagu0rx+vCRuDVcJaV/f9rjyu/YIEWCCHmLUOUvyUUivGVbUFuNQaDKe6Ed5S2PR7pq+KL85BQYIEWCCtOtMwKvLv0bVjz/ZYu6EUW318xWvmgzXAomDu0ge2YQ==:”;

It’s all specified in the HTTP gateway spec, but it’s probably easier to use this crate that does it all on its own: ic_response_verification - Rust

1 Like