Diode Message Storage coming to Difinity

Developers Getting Started
8

We just announced going live into production with the new zone canisters providing the encrypted message storage here: Diode ICP Zone Canisters going live

Diving into this technically the biggest lift here was to setup the right deployment strategy. After reading up here on the forum and some of the great work from the Cycleops people we decided to follow their Battery->Children pattern:

image
image875×363 14.7 KB

Again source code and architecture details are all available here:

The Good, (the open), the bad, the ugly

  • The factory pattern works great. Our app can autonomously create new children
  • Upgrading the children of the factory (and the factory itself) is not straight forward but ended up working great for us.

Open Questions

  • How to monitor a large fleet of canisters (1000s) effectively for resource usage patterns? What are other people using here?
    • Especially tracking stable memory consumption seems impossible from motoko or via status atm.
  • We ended up spawning new canisters with 1 trillion cycles (~1.30 USD) for every single canister - couldn’t make lower values work. This seems bit high for us especially knowing that many of these canisters will be in use only very shortly in cases when users churn out of the app. Can we reduce our losses here or recup the otherwise lost cycles?
  • Our factory is deployed on this subnet - does really mean that all children will be created there as well? That seems like an issue when we are going to create thousands of canisters. Should create multiple factories on different subnets and load balance ourselves or will the IC internals help us out?

Bad

  • It seems Denial-of-service (DOS) attacks on canisters are really easy to do because of the reverse-gas model. At least from motoko it didn’t seem like there is a good way of protecting against those. Specifically even checking for caller permissions costs gas and would be enough to run a DOS
  • There is really little documented information about the internals of the canister wasm upgrade process. We had learn a lot through trial and error, and many details are still unknown/unclear. (E.g. why is a motoko actor class constructor called again when doing mode=#upgrade?)

Ugly

  • Chain Fusion call-outs can timeout and many endpoints do not support IPv6 without additional work. We had to do some investigation and discussion with our endpoint providers to get it supported everywhere we needed.
  • ICP documentation often states “queries a free at the moment” but doesn’t go deeper into this. Were kind of afraid that this might change some day and suddenly render our solution non-viable from a cost perspective
  • Why are cycles pegged against XRP and not getting cheaper as hardware improves? E.g. ICP launched in 2021 and since the FLOPS/dollar has improved 8x but cycles cost stayed the same?
  • I once did a dfx cycles transfer <factory> <amount> instead of dfx cycles top-up <factory> <amount> and it took me quite some time to understand the difference and then be able to transfer them back and convert from “cycles ledger balance at the canister” to “cycles gas balance at the container” or whatever the right wording is for these two states. Also until now it’s unclear to me what magic the icp call dfx topup is doing to convert a cycles ledger balance to a cycles gas balance.
  • In order to use http out-calls it is necessary to supply a shared query as tranform function oracle_transform_function : shared query TransformArgs -> async HttpResponsePayload; – In motoko this kind of function is only possible on an actor (not inside a module) and so we had to expose this function on the most outer layer of our canister while the oracle and http interface logic itself is nicely encapsulated in a module. This felt like a really ugly leak of abstraction just because of a language/interface limitation

The new Elixir ICP Agent

Lastly during this last stage we released a new ICP agent in the Elixir programming language https://elixir-lang.org

You can fetch the new agent at GitHub - diodechain/icp_agent: Elixir ICP agent for interacting with the internet computer

That’s it for today, but feel free to ask any questions. I’m happy to answer here or on discord.

Cheers!

6 Likes