A Vision for Software Building and Distribution Based on the Internet Computer

General
, ,
1

Background

Even if we assume that open-source code is trustworthy and no malicious code is accidentally merged, while we can easily verify the integrity of the source, we still lack an effective way to confirm that the built software truly corresponds to that unaltered code.

There is a risk: if a developer is untrustworthy, they could surreptitiously modify some parts of the source code before building the software without anyone noticing.

Currently, our approach is to allow reproduction. We record build environment details or use container technologies to log which container was used so that anyone can reproduce the build and verify that the published software was built without modifications.

However, several serious issues remain:

  • Very few people actually reproduce builds for verification.
  • For most software, once built and released by the developer, no one attempts reproduction.
  • In rare cases, only one or a few individuals try to reproduce a build.
  • The vast majority of users do not care about this issue—even if a potential risk exists.
  • Even if a user cares, there may be no mechanism for me to know whether others have reproduced the build.
  • Knowing that a few reproductions occurred does not reveal whether those few parties conspired together.

It seems that the only certain solution is for every user to build the software from source themselves—but that is clearly impractical.

The main idea here is to leverage the Internet Computer for software builds to fix these issues, aligning perfectly with the vision of the internet computer, world computer. The internet itself act as the computer, and this computer should be able to build software.

Feasibility and Practicality

Deterministic Builds

Our current verification method relies on reproduction. If a build can be reproduced, it implies that building from the source is deterministic.

Turing-Complete Canisters

Canisters on the Internet Computer are Turing complete, enabling complex build processes to be executed on-chain.

Efficient Resource Utilization:

Under the current scheme, the reproduction process consumes computational resources on the client side. By executing builds on each node in a subnet (each acting as one reproduction), the consumed resources are exactly those that would otherwise be spent by individual users. This approach is highly practical from a resource utilization standpoint.

Enhanced Trust Through Multiple Subnets

Suppose a software build is performed on a subnet with 11 nodes, yet I remain skeptical about their security. In that case, one can easily use the same Builder Canister on a different subnet to reproduce the build, thereby increasing the overall trustworthiness of the software.

Beyond Canister Builds

  • Not only does building canisters on the Internet Computer provide trust, but any build toolchain that can compile to WebAssembly (WASM) can be ported over. For example:Blockchain smart contracts that target WASM could directly benefit from the Internet Computer’s builder canisters.
  • For non-WASM smart contracts (such as Ethereum’s Solidity-based contracts), the build tools could be compiled to WASM and then ported to a canister.
  • General software (e.g., OS kernel modules, popular open-source CLI tools, desktop applications) may require cross-compilation using a WASM-based builder canister.

Bootstrapping

The builder canister itself can be built using another builder canister, thereby establishing trust in the builder canister’s integrity.

Self-Writing-Building Internet

Under the SWI paradigm, a complementary “self-building” mechanism is also required. Following the writing process, the internet itself should perform the building process, thereby eliminating reliance on centralized infrastructure.

How It Would Work

Develop Specialized Builder Canisters

Create a suite of canisters dedicated to building software—such as a Rust builder canister, a Motoko builder canister, a TypeScript builder canister, etc.

Submit Build Requests

Users can send build requests to the appropriate builder canister along with the URL or network path of the code repository.

Integrity Canister

We can query a Canister to verify the integrity and correspondence between the source code and the resulting software build after compilation.

Potential Extensions

Dependency Caching

Frequently used builder canisters could cache dependencies, assigning different canisters to different dependency versions.

Dedicated Subnets

A subnet dedicated solely to builder canisters could be established.

Customized Build Systems

Develop a build system optimized for running within a canister. This system would shed the complexities of traditional build systems, offering simplicity, ease of maintenance, and optimized efficiency for canister environments—ideal for new projects.

Implementation Challenges

Complex Build Processes

Some projects have intricate build processes that rely on various shell or Python scripts and multiple command-line tools. (For instance, see this project for an example.)

Large Dependency Trees

Projects to be built might involve a vast number of dependencies.

Portability to WASM

Many existing build systems are not easily migrated to WASM or to the canister environment of the Internet Computer.

Insights and Considerations

Frontend Toolchains

Many frontend toolchains, typically implemented in JavaScript or TypeScript, can already be compiled to WASM. For instance, the TypeScript compiler is entirely written in TypeScript, which might allow for rapid porting of build systems to canisters.

Cranelift Backend

Leveraging the Cranelift backend from the Bytecode Alliance may offer a more streamlined and efficient solution for software that targets WASM as its compilation output (e.g., Internet Computer canisters).

Rustc in WASM:

There are ongoing discussions about running Rustc in a WASM environment, see this discussion Compilation to WASM? · Issue #722 · rust-lang/miri · GitHub , with significant contributions from developers like bjorn3 · GitHub.

Summary

The Self-Building Internet. The internet will build and distribute itself with inherent trust

While the work required is significant, in the long term, it appears entirely achievable and could have a profound impact on the world.

Others

The idea was originally post on Nuance: https://nuance.xyz/force/13232-434go-diaaa-aaaaf-qakwq-cai/a-vision-for-software-building-and-distribution-based-on-the-world-computer .
I believe this article has the potential to become a ‘white paper.’

Due to my limited knowledge of build systems and compilers—and a lack of strong personal interest to delve deeply into them—I hope others (e.g. Dfinity team) will bring this idea to fruition so I can simply enjoy the benefits of such an infrastructure.

This is all I can think of for now. Thanks to AI for helping me organize these thoughts; I haven’t carefully proofread them yet. I’ll update the article if I come up with more later. Feel free to discuss with me, point out my mistakes, or offer some advice.

About Me