A Vision for Software Building and Distribution Based on the Internet Computer

Background

Even if we assume that open-source code is trustworthy and no malicious code is accidentally merged, while we can easily verify the integrity of the source, we still lack an effective way to confirm that the built software truly corresponds to that unaltered code.

There is a risk: if a developer is untrustworthy, they could surreptitiously modify some parts of the source code before building the software without anyone noticing. Even if developers have no malicious intent, their computers could potentially be hacked, which would still create problems.

Currently, our approach is to allow reproduction. We record build environment details or use container technologies to log which container was used so that anyone can reproduce the build and verify that the published software was built without modifications.

However, several serious issues remain:

  • Very few people actually reproduce builds for verification.
  • For most software, once built and released by the developer, no one attempts reproduction.
  • In rare cases, only one or a few individuals try to reproduce a build.
  • The vast majority of users do not care about this issue—even if a potential risk exists.
  • Even if a user cares, there may be no mechanism for me to know whether others have reproduced the build.
  • Knowing that a few reproductions occurred does not reveal whether those few parties conspired together.

It seems that the only certain solution is for every user to build the software from source themselves—but that is clearly impractical.

The main idea here is to leverage the Internet Computer for software builds to fix these issues, aligning perfectly with the vision of the internet computer, world computer. The internet itself act as the computer, and this computer should be able to build software.

Feasibility and Practicality

Deterministic Builds

Our current verification method relies on reproduction. If a build can be reproduced, it implies that building from the source is deterministic.

Let’s discuss reproducible builds and code verification once again
Many of the conclusions in this discussion remain valid. We need to standardize how to achieve reproducible builds and provide templates. Therefore, this article is not opposing the discussion in the link but rather extending it further.

Turing-Complete Canisters

Canisters on the Internet Computer are Turing complete, enabling complex build processes to be executed on-chain.

Efficient Resource Utilization:

Under the current scheme, the reproduction process consumes computational resources on the client side. By executing builds on each node in a subnet (each acting as one reproduction), the consumed resources are exactly those that would otherwise be spent by individual users. This approach is highly practical from a resource utilization standpoint.

Enhanced Trust Through Multiple Subnets

Suppose a software build is performed on a subnet with 11 nodes, yet I remain skeptical about their security. In that case, one can easily use the same Builder Canister on a different subnet to reproduce the build, thereby increasing the overall trustworthiness of the software.

Beyond Canister Builds

  • Not only does building canisters on the Internet Computer provide trust, but any build toolchain that can compile to WebAssembly (WASM) can be ported over. For example:Blockchain smart contracts that target WASM could directly benefit from the Internet Computer’s builder canisters.
  • For non-WASM smart contracts (such as Ethereum’s Solidity-based contracts), the build tools could be compiled to WASM and then ported to a canister.
  • General software (e.g., OS kernel modules, popular open-source CLI tools, desktop applications) may require cross-compilation using a WASM-based builder canister.

Bootstrapping

The builder canister itself can be built using another builder canister, thereby establishing trust in the builder canister’s integrity.

Self-Writing-Building Internet

Under the SWI paradigm, a complementary “self-building” mechanism is also required. Following the writing process, the internet itself should perform the building process, thereby eliminating reliance on centralized infrastructure.

How It Would Work

Develop Specialized Builder Canisters

Create a suite of canisters dedicated to building software—such as a Rust builder canister, a Motoko builder canister, a TypeScript builder canister, etc.

Submit Build Requests

Users can send build requests to the appropriate builder canister along with the URL or network path of the code repository.

Integrity Canister

We can query a Canister to verify the integrity and correspondence between the source code and the resulting software build after compilation.

Potential Extensions

Dependency Caching

Frequently used builder canisters could cache dependencies, assigning different canisters to different dependency versions.

Dedicated Subnets

A subnet dedicated solely to builder canisters could be established.

Customized Build Systems

Develop a build system optimized for running within a canister. This system would shed the complexities of traditional build systems, offering simplicity, ease of maintenance, and optimized efficiency for canister environments—ideal for new projects.

Implementation Challenges

Complex Build Processes

Some projects have intricate build processes that rely on various shell or Python scripts and multiple complex command-line tools. For example, frontend project rely on Node.js, vite, typescripts, etc. And Rust project rely on rustc (and llvm), cargo, etc. And Build tools may need to make a large number of HTTP requests, since projects to be built might involve a vast number of dependencies.

Some other concerns I have are that the build process will undoubtedly require executing a large number of instructions, but update calls have instruction limits.

Portability to WASM

Many existing build systems are not easily migrated to WASM or to the canister environment of the Internet Computer. For example, this includes using the IC API to make HTTP requests and utilizing stable memory to cache built objects.

Some Progress

Frontend Toolchains

Many frontend toolchains, typically implemented in JavaScript or TypeScript, can already be compiled to WASM. For instance, the TypeScript compiler is entirely written in TypeScript, which might allow for rapid porting of build systems to canisters.

Node v22 was successfully compiled to wasi environment and runs good with all features in wasmer, see this discussion compile NodeJS to wasm · Issue #3774 · nodejs/help · GitHub , with significant contributions from developers like cheplv (Jurijs Cerepanovs) · GitHub.

Rustc in WASM:

There are ongoing discussions about running Rustc in a WASM environment, see this discussion Compilation to WASM? · Issue #722 · rust-lang/miri · GitHub , with significant contributions from developers like bjorn3 · GitHub.

Leveraging the Cranelift backend from the Bytecode Alliance may offer a more streamlined and efficient solution for software that targets WASM as its compilation output (e.g., Internet Computer canisters).

Shell Scirpts in WASM

See GitHub - wasmerio/webassembly.sh: Open-source and installable PWA terminal powered by WebAssembly, WAPM, and Wasmer-JS đŸ–„

WASI in IC

Thanks to GitHub - wasm-forge/wasi2ic: WASI dependency replacer for the *.wasm files

Summary

The Self-Building Internet. The internet will build and distribute itself with inherent trust

While the work required is significant, in the long term, it appears entirely achievable and could have a profound impact on the world.

Others

The idea was originally post on Nuance: https://nuance.xyz/force/13232-434go-diaaa-aaaaf-qakwq-cai/a-vision-for-software-building-and-distribution-based-on-the-world-computer .
I believe this article has the potential to become a ‘white paper.’

Due to my limited knowledge of build systems and compilers—and a lack of strong personal interest to delve deeply into them — I hope others (e.g. Dfinity team) will bring this idea to fruition so I can simply enjoy the benefits of such an infrastructure.

This is all I can think of for now. Thanks to AI for helping me organize these thoughts; I haven’t carefully proofread them yet. I’ll update the article if I come up with more later. Feel free to discuss with me, point out my mistakes, or offer some advice.

About Me

About “and a lack of strong personal interest to delve deeply into them”,
let me tell you a secret: I have an even more important idea that I need to bring to life.

But if no one ends up doing it, I’ll probably give it a try. It is most likely to happen in the distant future, so don’t get your hopes up too high.

2 Likes