Community Consideration: GPU Use Cases and Specifications

Interesting proposition @jeshli
At first glance the main utility for GPU processing on-chain would be for AI training done within a secure and transparent environment.

From our (UnfoldVR) perspective, when it comes to gaming or intensive graphics you typically defer to the end user’s hardware to act as the graphical processor of whatever you’re trying to render. Either the browser or the PC or the mobile device renders the assets downloaded from ICP canisters and reads/writes instructions to the blockchain.

The main utility from a gaming perspective would be strictly from running an AI algorithm on-chain to do interactive chat on characters or other similar tasks.

One potentially relevant use case would be having GPU machines that would do pixel streaming (as Epic Games references it) or anything similar to cloud real-time rendering and then stream that to the end-user using WebRTC protocol. In this scenario the user has a WebRTC connection to the GPU machine. The user makes an action and the GPU would run a render engine which would render, in real-time, locally, a scene, based on User’s input (movement, configurations, selections, etc.). Then it would stream that render back to the user. This technology is similar to Xbox Cloud or Nvidia GeForce Now

In practice, as most recently demonstrated by Xbox Cloud support on the new Quest 3 VR headset, this isn’t a great solution for fast paced games/experiences. Even on a traditional architecture, even with Wifi 6 and high internet speeds, getting user input, rendering a high quality scene on a server machine, compressing a video frame, sending that over and finally decompressing it locally introduces at least 0.5 seconds lag.

As such, I think it’s optimal to have the rendering burden on the end-user and the graphics optimization burden on the developers, for graphics intensive scenarios. As devices become more and more powerful and miniaturized, the use cases for cloud real-time rendering are very niche.

Finally, there are multiply use-cases for non-real-time rendering solutions (web based product design, architectural design, etc). You could render real-time, on-device, a simplified version of the 3D scene and process a high fidelity render on-chain, on demand. But this could also happen on the CPU. And the use cases I can think of don’t warrant the level of security the blockchain provides.