Video (streaming) optimization and embedding

[integral_wizard]

Hi.

I’m looking forward to storing and embedding videos on multiple ICP websites. The challenge is to make it performant and customizable for different devices and browsers. I feel like this should be a rather common challenge for the ICP ecosystem as more and more social media content goes to video formats. I would like to not reinvent the wheel with my implementation and would ideally use some out-of-the-box solution.

There are various dApps that use videos.

• PortalDapp is perhaps the most famous and promising but their Twitter has been silent for over 6 months and their discord link is broken. I didn’t use it and at this point am cautious ofusing them.

• Hot or Not stores a lot of videos but I am not too knowledgable whether they store them 100% on-chain and how are the optimizing the UX.

• Civol.ai uses video format for shared sensemaking. Am interested to know more about that.

• There are various file folders/ personal asset canisters that look promising but very beta and so not sure of the future. Also, they are not necessarily optimized for video.

• I have heard of multiple other dApps that used video but at those times I didn’t need the video streaming myself and so didn’t look further. Now I do.

I would like an on-chain self-custodial solution (owned by the webapp canister). I have a hunch that there are different pieces or even full solutions out there. Does anyone know more? How could we make sharing video across ICP as seamless as possible? This (incl AR/ VR) is the medium of the now and the future.

[Vivienne]

You found one of the big areas that are still under construction I don’t know of an ideal solution, but I can give you some pointers about the asset canister.

AFAIK most services use the asset canister that is bundled with dfx when you choose canister type assets. It is entirely self-custodial and is probably the most stable thing out there for now, but it still struggles with ‘large’ amounts of data (say >1.8GB of data in one canister), and streaming assets is not well supported. I don’t even know what happens if you simply try to load a 400MB video into it. It’s probably best if you simply try it

The other important thing you should know about the asset canister is that for anything that’s larger than a single message on the IC (2MB) certified (or ‘verifiably correct’) responses are not implemented yet. Because of that, your assets will only be available via raw.icp0.io and not via icp0.io I wanted to start a few months ago but then got pulled into more urgent stuff. If you would be willing to work on it (entry point is here if you want to have a look) I’d be very happy to help you with the implementation of certified streaming

[integral_wizard]

Thank you for those thoughts.

I’m wondering how the various dApps have made this work? Some are I guess off-chain. But others claim to be on-chain. It would bring a lot of value to be able to leverage the work on the complexities of video optimization and not reinvent the wheel everywhere.

[acludo2]

I’ve been working on a video solution using CanDB as my video canister and store the embeds as chunks of less than 2mb so i can reconstruct the videos in the front end.

i been trying to get the MSE js utility to work so that i can create streaming and not have to wait for my video to load all the chunks. probably just a codec format issue and not the data itself why MSE not working.

but yeah i think the biggest issue is probably canister size and indexing while managing a large video library would probably become nightmare.

[skilesare]

I’ve been thinking a lot about this and I think there is a solution similar to the service worker. When you request a video, the gateway interprets if you are requesting the file or if the file is being requested from a web page.(maybe the referer header?). If it is a request from a known browser and it was not embedded in a page, then you can serve a simple HTML page with video.js as the only thing that serves up the chunks of your video. If it is a request of the actual file(referred from a page - probably embedded) then it can concatenate the chunks and serve it as one big file.

We have a plan to do something like this from the ORIGYN NFT, but it is a bit buried on our todo list.

[Sormarler]

Video optimization and embedding is clearly an important challenge that needs to be solved for the Internet Computer to reach mass adoption. Video is becoming the dominant form of content online. As more apps and services move to the IC, they will need performant video support to compete with traditional web apps. This is especially true for social media, entertainment and metaverse types of apps that rely heavily on video.

There are already several examples of apps trying to use video on the IC like PortalDapp ( now gone), Hot or Not( videos not the IC, and Civol.ai but running into limitations. The video support needs to improve to unlock the potential of these innovative apps. Large videos and files are a known challenge for blockchains generally. The IC has an opportunity here to develop an elegant solution for self-custodial, on-chain video storage and streaming. This would highlight its advantages over traditional blockchains lacking in this area. Also, each node on the IC has over 30 TB of unused storage space. What are we doing to improve these inefficiencies?

Without better video handling, developers will be hesitant to build media-rich apps on the IC due to UX concerns. And consumers won’t adopt apps that have a poor video experience compared to traditional apps. Video streaming at scale needs to work smoothly across devices and browsers. A fragmented video experience will quickly turn users off.

Seamless video support is a must-have for the IC to achieve mainstream adoption and compete with traditional platforms. The community recognizes this as an urgent gap that needs dedicated focus. Solving the technical challenges around video is key to unlocking many exciting use cases and applications.

[Infeelit]

I’ve been down a similar road with my own projects, trying to make everything smooth across different devices and browsers. To make things a bit easier, I found this movie maker super handy tool for editing my videos before even thinking about how to embed them. It’s kind of like a modern take on the old movies vibe, but really easy to use and great for getting your videos ready for the web. It helped me a lot, especially with making sure my content looked good no matter where it was played.

[lastmjs]

For those interested, we at Demergent Labs released some demo code of using range requests to perform video streaming.

Here’s the live demo: https://f2j4a-eiaaa-aaaam-ab7nq-cai.raw.icp0.io/

Here’s the original tweet: https://twitter.com/lastmjs/status/1765877758117216467

Here’s the source code: azle/examples/audio_and_video/src/backend/server.ts at main · demergent-labs/azle · GitHub

Using Azle and range requests makes video streaming incredibly simple, and is supported by the HTML video element automatically.

[infu]

Streaming with static chunked files solution.

Use HLS. No need for Dash.
It’s not supported in all browsers if you put it inside the video tag.

But with hls.js using Media Source Extensions, there will be ~ 3% of old browsers that can’t play it.

Best use video.js which uses hls.js under the hood and makes it easier.
To support the other 3% you can leave the unchunked mp4 in sources as well.

video.js

Convert files:

ffmpeg -i input.mp4 -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -codec:v libx264 -codec:a aac -f hls -hls_time 10 -hls_playlist_type vod output.m3u8

You can also convert to multiple resolutions. The player will start with the lowest one and stay there if the connection isn’t good or it will increase the resolution.
The IC limits won’t be a problem with chunked files, you can have 4gb 8k video if you want.

[infu]

If you really want to make it fly.

Here is the H264 support, that’s what you use by default

But you should also convert files using H265 video codec which will make them 2 times smaller.
Perhaps provide both h264 and h265 for better compatibility.

ffmpeg -i input.mp4 -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -codec:v libx265 -crf 28 -codec:a aac -f hls -hls_time 10 -hls_playlist_type vod output.m3u8

[integral_wizard]

Amazing. THIS IS IT. Very elegant solution to remove limits for streaming video onchain. Thank you very much! Can see an example on why-internetcomputer.org

[infu]

Good video for testing:
[https://durian.blender.org/]
4.2GB 4k free to distribute video.

Tip: just ask chatGPT to write ffmpeg commands otherwise they are pretty complicated.

H264, H265, VP9 - video codecs
AAC, MP3 - audio codecs
HLS, DASH - media streaming protocol
MPEG-TS, WEBM - transport stream

If we pick HLS and h264 than that’s okay, but if we push for a better codec, then things get hard.
We will have to encode the video into two different steams to support all browsers. (maybe even more because of audio codecs)
dash-webm-vp9 and hls-mpeg-h265

Tip: There is a ffmpeg wasm that can run in web browsers. A browser can convert and upload any video into the formats needed.

[integral_wizard]

So you mean we could implement the ffmpeg command in a canister so anybody could convert their video into proper formats using UI? With perhaps frontend code provided to use in code. Would be a nice hackathon/ weekend project.

[skilesare]

This is very unlikely to be possible without significant effort of rewriting the encoder to be"Chunkable". A video of any size would rapidly hit the cycle limit inside a canister.

You’d need to fork the library and create an encoding loop that was able to stop and resume across different rounds.

About three years ago, I started writing some bitmap transforms and rapidly gave up when I realized I did not have the skills to do that. Maybe it would be a bit easier now with AI, but it would be a significant undertaking still, and it wouldn’t be that fast.

Unless you have the use case of needing a smart contract to certify that a transformation of a video followed an explicit process, it’s much easier to process the content off chain with existing libraries and then upload it.

I think @infu 's comment about wasm was more focused on being able to easily include it in your web dapp So that you could encode any video your user might have right on the client side before uploading.

[infu]

Yes, I meant ffmpeg in a browser. Demo here: [https://ffmpegwasm.netlify.app/]
code: [GitHub - ffmpegwasm/ffmpeg.wasm: FFmpeg for browser, powered by WebAssembly]

This would allow a dapp to convert and upload web-compatible video files without relying on off-chain conversion services. Usually mobile and desktop devices produce videos that can’t just be uploaded to the web and require conversion.

[officex]

Does anyone know the current state of this? Our team is looking for an up-to-date sample repo for rust to work off

[SummerHouse]

Are you @skilesare or @infu able to point to docs on how to store off chain then run that through the canister?

Right now, I have two JS functions. The one I have for a photo album that uses a flip book function works great, however I’ve found it randomly not populate and in turn displays all of the photos right after each other. Other times it works flawlessly. Images are not the main issue causing my delay…

It’s the video functionality…

Right now my video player works something like this:

/**
 * VIDEO GALLERY FUNCTIONALITY (HLS + hls.js)
 * 
 * Builds an adaptive HLS video player optimized for deployment on the Internet Computer (IC).
 * Utilizes the HLS (HTTP Live Streaming) format for efficient, chunk-based video delivery.
 * Leverages the hls.js library for broad browser compatibility and stable playback.
 * 
 * Features:
 * - Seamless HLS playback across all major browsers (Safari native, hls.js for others)
 * - Previous/Next navigation controls
 * - Clickable playlist sidebar for quick video selection
 * - Video counter display (e.g., “1 / 5”)
 * - No autoplay — playback begins only when the user presses play
 * - Optimized for boundary node asset delivery on the Internet Computer
 */

Before I remove this functionality (stream via chunking), I want to see if this is the preferred method.

Or, is what @skilesare is saying that you upload and host video files on other cloud/decentralized platforms like Cloudflare or Filecoin (which is not a great project/solution imo), and then reference that video’s live public URL in the source code — rather than uploading them to the canister as static files that get rendered directly from the canister and to the browser?

If that is what you are suggesting, then what’s a good way to not store URL information in a way that’s facing the client or being viewed if I open-source the codebase?

I can send direct code/structure of the function if needed. I might need to reach out via Discord or some other method though.

And the file size is only 13 MB – 17 MB…

At first it was working as an .mp4 file in Chrome but then broke in Safari. Then it worked in Safari and not Chrome. So, I tried converting to H265 and streaming with static chunk files, and now the video just does not load in the browser — even using raw URL…

Background on why I’m bringing this solved post/ What I’m doing:

My intent here is to host completely “on-chain,” even the small video file, then make explicit documentation and a walkthrough guide on how to take a standard portfolio — like one off Replit or other vibe-code platforms — for the non-technical individual or new student.

My hope is that I can demonstrate a fairly quick solution to offset hosting costs for these simple (non-dynamic) static sites and make them more secure. This way, users of these platforms can use the Replit or “vibe-code” style IDEs and sandboxes as a pretty good “test net” or local development.

At the very least, it’s a decent interactive learning tool to help ease in the newer dev completely overwhelmed by both the IC tech stack and traditional centralized platforms like AWS, Replit, Render, Firebase, etc., etc….

At the moment, one month of hosting costs on other cloud-based platforms can run around the same as a few months of hosting through a canister. As these expenses continue to rise, and subscription-based platforms increase their monetization of these services, users are going to eventually look for all offsets to help annually — rather than making “short moves” (1 day – 11 months).

Being able to develop an MVP with the help of AI tools like “AI Coding Agents” on different tech stacks, while structuring the app with the IC in mind, will allow for an optimal experience — one that actually enhances web2 vibe coding or sandbox IDEs rather than competing.

After they make their portfolio (most devs do), they can quickly use this documentation to deploy a simple static portfolio app and quickly understand how the IC tech stack works in comparison to those around it, and in web2.

[skilesare]

Safari is a PITA when trying to do this stuff.

For any video it first requests 0 bytes so that you return a header with the full file size and the it starts dynamically requesting chunks. This has been an issue since the ic certifies in static chunk sizes.

This is likely why you are seeing odd behavior in the two browsers.

@yrgg would be able to tell you more as he’s forgotten more about this topic than I’ve ever known.

[yrgg]

Yeah I remember Safari being annoying. One of the issues is the video player will be blocked if you have autoplay on, but in the snippet you posted it looks like it’s disabled. You said you’re using h265 codec which I had trouble getting to work, I don’t remember why tbh. You might have better luck switching to h264 though.

[SummerHouse]

Yeah, I removed the chunking method and served as a single static .mp4 and it works great on chrome. Now safari it just won’t load lol

Ugh this debugging is going to take forever lol If I pinpoint the exact issue with it on safari I’ll lob that up here in case the next person with this issue stumbles into the post and has the same issue…

Out of curiosity is this what caused so many road blocks when doing portal?

FWIW At the time I had absolutely no idea how difficult this actually is, was, will always be (even with A1 Sauce).

I also now understand how you’d prevent “negative use of the services”. At least in ways that originally I was naively concerned about well intended or not… and now I see there’s a limited chance of it occurring right now, and there’s some options to prevent it…

At any rate I needed to not need to worry about it, and for the noise I created around the project I apologize. I wish you the best of luck on it moving forward as a decentralized Netflix is needed in today’s world.

---

PS If there’s further progress dealing with this restraint on the IC and people come across it/ and this pst please let me know.

Thanks again!