I’m a strong believer in adressing content through its hash like in IPFS, as it allows clients to verify the integrity of content no matter where they fetch it from. This makes caching trivial, allows you to scale serving with CDNs without losing any trust guarantees, and you also don’t need to trust ICP’s root key. Furthermore, UnixFS provides a standard for splitting large files or directories into Merkle DAGs with small block sizes, ideal to work around ICP’s 2MB message limit.
I put together a quick demo of serving images and videos using stable memory as a block store and a service worker that fetches the necessary blocks and verifies integrity:
(if it doesn’t work on first try, try refreshing, I might still have a race condition in my service worker logic. Also, seeking through the video works fine in Chrome, but not Firefox for me)
I think there are a ton of things you could build with this:
Horizontally scalable storage. Just need an index cansiter that could serve a 302 redirect to the canister that has the block, and that deploys a new canister when needed
It’s a great format for transfering large amounts of data between canisters
It’s also a great format for copy-on-write. Want to change one byte in an archive that’s terrabytes? No problem, just requires log(n) new blocks instead of writing a whole new file
You could build a (centralized) bridge node that makes blocks from ICP canisters available to the wider IPFS network
Depends on what you mean by “on ipfs”. The problem with IPFS is that it doesn’t actually provide you with storage and you usually need to pay a central provider to pin IPFS content for you. That’s not an option for a true dapp that can’t store API keys. So the selling point of this demo is actually that it’s not on any existing IPFS storage provider, but stored on ICP without the need for any API keys or central provider.
If you’re talking about IPFS as a retrieval network, then no, you can’t just fetch the ICP blocks via any IPFS gateways. Since ICP can’t speak the p2p protocol, you’d need an off-chain IPFS node that relays the blocks from ICP. But that shouldn’t be too hard to build if you need it and it wouldn’t introduce more centralization than IPFS has otherwise (where most nodes will block bad bits).
That said, if you’re hosting content for your own app, you know where it’s located and can just use the ICP HTTP gateway as an IPFS gateway (as I did in my demo), no need for a bridge.
Thoughts on redstuff?
Haven’t come across that before, is that this Walrus protocol? Sounds similiar to Arweave? Arweave would certainly still be cheaper than ICP from my understanding (~$5/GB forever rather than $5/GB/year), so getting data from ICP onto Arweave would also be an interesting demo. That said, storing data on ICP directly certainly has its advantages, especially if your dapp is already running on ICP. The pay-as-you-go model rather than pay upfront might be cheaper for some use cases and I think ICP’s HTTP gateways are a lot more generous when it comes to actually serving content.
