TL;DR IC internals: orthogonal persistence
Orthogonal persistence was the first feature I worked on at DFINITY. I still have warm feelings for that feature; I would love more people to understand it deeper. The article is not beginner-friendly; it targets experienced developers who want to dig deeper into the tech.
Would you like to see a similar technical article on some other aspect of the IC? Answer in a comment.
Big thanks to @ulan for reviewing the draft of the article.
Very interesting read, would love to see more.
Do you already have an idea how big files will be stored? Will they always be loaded in memory?
I assume you mean checkpoint files. Checkpoint files are memory-mapped, which allows the kernel to be clever about loading pages into RAM. Only actively used pages need to occupy physical memory, and the kernel can always reclaim those pages because there is a copy on disk. This approach scales to terabyte-sized states.
@roman-kashitsyn Fantastic blog. I especially enjoyed your post on replicated state machines. Do you still plan to do a post/posts on the topics of “how different subnets communicate with one another, how individual worker bees form the swarm”?
as of April 2022, 95% of message executions change at most seven memory pages.
Uh, real numbers, exciting!
I wonder how many of them only have 7 non-static memory pages to begin with. But that’d be slightly harder to measure.
Such memories open up attractive optimization opportunities, but we did not have enough pressure to implement them.
Sounds like you are sad that pressure wasn’t high ehough 
Thanks @roman-kashitsyn , always a pleasure to read your blog posts.
This one was super interesting as well, thanks for sharing, it is much appreciated !
Andreas also proposed short-term memories with a lifetime of a single message execution. Such memories open up attractive optimization opportunities, but we did not have enough pressure to implement them.
Could you elaborate on this one please ? What kind of optimization you have in mind ? What kind of pressure would have been useful in this case ?
Cheers
One of the benefits of short-term memory is that the runtime doesn’t need to track dirty pages, so this memory might be measurably faster to use.
The Motoko runtime could use the short-term memory as fast minor heap. Rust canisters could use it as a scratchpad with a fast bump memory allocator.
If the Motoko team was struggling with improving their GC algorithm and short-term memories was the easiest way forward, we could have implemented this feature. Currently, there are plenty of other low-hanging fruit.
Yes, I like the idea a lot. However, it’s definitely not the most important feature to work on right now.
Great article, thanks for sharing!
A very basic question: what happens if a replica crashes (or node goes down) in between checkpoints? If the in-memory running page deltas are lost, how does the replica recover the actor memory snapshot after it restarts?
There are two recovery scenarios:
Nice! Another article to add to my IC reading list.