A piece by Jack Clark (one of Anthropic’s co-founders) on Substack
https://importai.substack.com/p/import-ai-439-ai-kernels-decentralized
where I focus on this bit
Decentralized training is getting better very quickly - which has major policy implications
with Clark asking
Could a decentralized AI training run ever rival the compute of a frontier training run? Probably not. But could decentralized training runs get far larger and support the development of more capable models developed by a much larger collective than just the frontier AI companies of today? Yes.
This is quite an interesting take, condensing his view on the findings of another paper by EpochAI he cites.
He then makes the following statement.
Fundamentally, decentralized training is a political technology that will alter the politics of compute at the frontier.
As a researcher (and entrepreneur) myself, I believe that this is a subject that deserves some deeper debate (maybe there is one, and I’m just ignorant about it) within DFINITY. It’s about looking at an objective technically well aligned to the tech stack and proposition behind ICP (whether its economics or current organization make it feasible is a separate, but key, discussion) with an upside that goes beyond tapping into more commoditized (due to it quickly becoming a saturated space) consumer-facing proposition, such as Caffeine. Don’t get me wrong, this is not about taking merits away from Caffeine, but exploring (complementing with) the high-end AI spectrum in the decentralized space, given the technology stack.
If guys like Clark are talking about it, perhaps worth paying attention
Maybe the economics don’t add up now, but think it’s a worthwhile debate to have.
4 Likes
Even assuming future Gen-3 nodes on the Internet Computer were equipped with GPUs, it remains unclear how GPU-based computation could be integrated into ICP’s deterministic execution model.
ICP’s consensus relies on the assumption that all replicas in a subnet, given identical inputs, produce bit-identical state transitions. This requirement is fundamental to safety and finality. GPU execution, however, is inherently non-deterministic due to parallel scheduling, floating-point behavior, hardware variance, and driver-level optimizations. These properties persist even when restricting kernels, fixing launch parameters, or using so-called deterministic modes, which at best provide reproducibility on a single machine class rather than across independently replicated nodes.
Given this, it seems difficult to reconcile direct GPU execution with ICP’s replicated Wasm execution without weakening determinism or introducing new trust assumptions. Even homogeneous hardware would not eliminate divergence caused by low-level execution differences, and a single bit mismatch would be sufficient to break consensus.
This suggests that, even with GPUs available at the node level, they cannot realistically function as part of the deterministic execution layer. Instead, GPUs would have to be treated as non-deterministic coprocessors operating outside consensus, with their outputs verified, sampled, or economically enforced by deterministic canister logic.
The open question, then, is not whether GPUs can exist on ICP nodes, but rather what architectural role they are expected to play if determinism remains a hard constraint: execution, or coordination and verification. The latter appears more consistent with ICP’s design principles, positioning GPUs as off-consensus compute resources whose results are governed and validated on-chain rather than directly executed within the replicated state machine.
2 Likes
Get exactly what you mean, the issue is whether you need consensus-driven execution for AI inference or not, or whether to limit it to end-point delivery. So the Q is more about whether you can integrate decentralized AI as a layer within an otherwise consensus-driven architecture, and what role (if any) would consensus play. It’s an engineering challenge, no doubt, but there is already an infrastructure in place.
At the very least, it all prompts provocative thinking 
1 Like