Introducing MulicanisterOps: helps to build, test, deploy and scale multi-subnet complex applications

[akup]

Dear ICP Community, we would like to introduce DevOps infrastructure project MulticanisterOps, that aims to help to build, test, deploy and scale multi-subnet complex applications.

Project is in development phase and we are introducing main goals of the project to get community feedback and feature requests to add to roadmap.

Motivation

We are building complex application that uses nearly all IC specific features combining onchain and offchain functionality. And far before release on the step of architecturing the app we want to solve future scaling bottlenecks.

According to story of IC app with a lot of users described here, we see that it is not so easy to scale on IC, but certainly possible. And it could be thorny path to evolve from single-canister architecture to multi-canister and multi-subnet systems. And a lot of mistakes could be done on this path.

Also it is crucial to have clean and easy development flow, when developer can develop and test multi-subnet system locally, push it to development branch and test it with other team developer’s modules in the cloud, and then push to test branch where system will be automatically deployed and ready for testing (all without submitting to mainnet). After tests are complete, release could be created and github workflow will push it to mainnet where it should be approven by DAO to make updates decentralized.

All of it certainly could be done, but there is no any step by step best practice guide to acheave it.

So we start thinking on infrastructure project that would help to acheave our internal goals and could help other teams to build more complex apps on IC easier.

Overview

ICR (Internet Computer Registry) is an infrastructure platform for orchestrating and managing the application lifecycle, acting as something like a Kubernetes for multi-canister environments on IC. The project aims to radically simplify and automate the CI/CD processes, deployments for tests, log-tracing, versioning, shipping to mainnet and scaling processes for complex multicanister IC applications. It also provides a unified test environment for onchain (possible multisubnet) and offchain components that use http-outcall functionality.

It can and will provide an easy way for developers to build scalable complex IC solutions with combination with offchain functionality.

How It Works (should work)

Development Workflow

1. Mono-repository Structure: Developers create a mono-repository that contains canister modules and backend offchain modules

2. Declarative Configuration: Using declarations, these modules are combined into an application that can be built and deployed to cloud-based development environments

3. Cloud Infrastructure: GitHub workflows help create Kubernetes clusters in the cloud (AWS, GCS, etc.) where Pocket IC, offchain modules, and logging/monitoring tooling are installed

4. Orchestration Core: At Pocket IC, orchestration core canisters are automatically installed

Deployment Process

• CLI Tools: Built application modules and canister WASMs are uploaded to the orchestration core

• Registry Management: Canisters are added to the registry with branch and tag metadata and created in Pocket IC

• Service Discovery: Orchestration helps discover application canisters and offchain modules as services, enabling multiple non-conflicting versions of the same application in a cluster for testing and development purposes

• Mainnet Deployment: After testing, applications can be released to mainnet with project GitHub workflows

• Auto-scaling: Orchestration helps autoscale canisters (replicate and shard) across subnets

CI/CD Scheme

Roadmap

We’ve added BUIDL to WHCL25 hackaton (upvote the project if you like) and want till October implement main functionality and are going to provide an example and describe the development and configuration of an application that has:

• Multiple canisters communicating via service discovery

• Offchain module integration via http-outcalls (also connected via service discovery)

• Complete build and deployment workflow using GitHub workflows

• Multi-version deployment for different test environments

• Mainnet deployment with DAO-governed updates

• As a bonus github workflows to create dev/test clusters in Google cloud or AWS

Github link

Will be updated significantly in a few days:
https://github.com/akup/multicanister

Feedback

Looking forward for community feedback on project goals, and feature requests

[akup]

We are continuing to develop MulticanisterOps project, and we are going to present example app built with the tool and step by step guide how to use it. Approximate date is late september - early october.
Brief description is here http://mc-ops.tech

This month we’ve added DAO implemented with SNS, to control redeployment of the app. Main thing, that if your app is complex and has multiple canisters or miltiple subnets the tool will make it possible to update the app only with DAO approval and update it as a whole on app.

We’ve adding evm log tracing and instrumentation support. In one infrastructure developers can build ICP parts, offchain parts and EVM (subject to extend) parts. But during debugging developers and testers can see all logs and logs tracing with Grafana Loki and Grafana Tempo. It makes it easier to develop complex multichain apps with offchain components, and make request tracing transparent.

Also we are adding ICP support of Opentelemetry requests instrumentation. It is useful for debugging on development stand. Earlier we’ve added devops infrastracture to run ICP apps in a cloud and gather logs to Loki. Our goal is to make fully instrumentated infrastructure with ICP, EVM and offchain components.

Everybody who are interested in the project. Please stay tuned and add some stars to github page GitHub - akup/multicanister
Your feedback, when first release with example app is ready, will be very important for us.