(was hidden by ZackDS/Wenzel crew as they said it was advertising lol)
The IcyDB 0.204–0.213 Roadmap
I’ve completed IcyDB 0.204 and have now started work on 0.205.
At this point, IcyDB’s core SQL feature set works and is largely complete for its intended scope. The next releases are about tightening the architecture, removing unnecessary work and adding the remaining durability and integrity guarantees needed before 1.0.
These releases are deliberately ordered: measure first, optimize second, then strengthen writes, constraints and persisted data.
0.204 — SQL correctness matrix and generative oracles
I built a large, automatically generated matrix of SQL queries, data shapes and edge cases.
An independent oracle calculates the expected result for each scenario, and I compare IcyDB’s output against it. This lets me test thousands of query combinations that would be impossible to maintain as handwritten tests.
The same system also records:
-
the route selected by the planner;
-
the work performed by each execution phase;
-
the number of rows read, decoded and returned;
-
instruction costs at different database sizes; and
-
cold versus warm plan-cache performance.
This gives me a reproducible map of both SQL correctness and performance.
In short, 0.204 tells me whether IcyDB returned the right answer and how much work it performed to produce it.
0.205 — Grouped early materialization
0.205 addresses a class of expensive ordered GROUP BY queries.
Previously, IcyDB could retain the state for every group before returning a bounded result. That is unnecessary when the selected access path proves that rows are already arriving in the required group order.
IcyDB can now finalize a completed group earlier and release its working state instead of retaining every group until the end.
For example, if a query only needs the first ten ordered groups, IcyDB should not retain hundreds of completed groups unnecessarily.
The SQL result remains identical, but eligible queries perform less materialization and retain much less state.
0.206 — SQL performance remediation
0.204 found and measured IcyDB’s expensive query families. After the grouped-query improvement in 0.205, I will rerun the complete profile and rank the largest remaining costs.
Each expensive family will be classified as:
-
work inherently required by SQL semantics;
-
work that requires a better application index;
-
avoidable work inside IcyDB; or
-
suspected work that needs better measurement.
I will then fix the largest proven engine waste at its general architectural owner.
This will not introduce query-specific benchmark tricks, hidden indexes or a cost-based optimizer. The goal is to remove unnecessary decoding, materialization, copying, planning or traversal while preserving exactly the same SQL behaviour.
0.207 — Redo-only schema and index publication
Schema and index changes must have one durable source of truth.
0.207 will make the committed redo record authoritative for publishing these changes. This prevents schema or index state from becoming partially visible if an operation fails or the canister traps halfway through.
The intended rule is simple:
A schema or index change is durably committed and visible, or it did not happen.
There should be no second publication path capable of disagreeing with the committed database state.
0.208 — Exact composite contracts and explicit AnyValue
0.208 will tighten the contracts for composite values such as records, lists, maps and nested values.
Their accepted schema shape, encoded representation and decoded value must agree exactly. I do not want several parts of IcyDB independently interpreting what the same composite field means.
I am also considering an explicit AnyValue type for fields that genuinely need to store arbitrary values.
The important part is that flexibility remains deliberate. A normal IcyDB field stays strongly typed, while an application must explicitly choose AnyValue where unstructured data is required.
I am still considering whether AnyValue is worth including and how to prevent it from weakening the rest of the type system.
0.209 — Temporal defaults and versioned row layouts
0.209 will add properly defined temporal defaults, such as database values generated from the current transaction time.
It will also version the physical layout of persisted rows.
When IcyDB reads a row, it should know exactly which layout produced it rather than interpreting old bytes using only the current schema. That becomes increasingly important as schemas evolve.
This gives future migrations and row-format changes a clear foundation without making persisted data ambiguous.
0.210 — Exact and resumable bulk updates
Large updates can exceed the Internet Computer’s per-message instruction limit.
0.210 will let IcyDB divide a bulk update into deterministic, bounded batches and resume it safely across multiple calls.
For example, Toko might need to execute:
UPDATE tokens
SET status = 'Live'
WHERE collection_id = ?;
A collection could contain thousands of tokens. Instead of attempting the entire update in one call, IcyDB could update 200 rows, return resumable progress and continue with the next batch when called again.
Repeated calls would eventually complete the exact operation without exceeding the IC limit or repeatedly updating rows that were already completed.
This is one of the features I am most excited about because it makes ordinary database-style bulk operations practical within the IC execution model.
0.211 — Accepted-catalog constraints
0.211 will make database constraints part of IcyDB’s accepted schema catalog.
The accepted catalog is the database’s authoritative understanding of the schema. Constraint enforcement should come from that authority rather than being scattered across application code or interpreted differently by separate runtime paths.
This is part of the transition from “typed storage with SQL” into a database that owns and enforces its declared rules consistently.
0.212 — Bounded and resumable integrity checking
A complete database integrity check may also be too expensive to finish in one IC message.
0.212 will allow IcyDB to inspect rows, indexes, constraints and catalog relationships in bounded batches.
Each call can return resumable progress so the next call continues from the same position. Eventually, the entire database can be checked without exceeding an instruction limit or restarting from the beginning.
The final report should identify exactly what was checked and provide actionable evidence for anything inconsistent or corrupt.
0.213 — Exact unsigned identity generation
0.213 will add database-generated unsigned integer identities—essentially a carefully defined IcyDB version of AUTO_INCREMENT.
Not every table needs a UUID. Sequential numeric identities are often a better fit for:
-
logs;
-
events;
-
internal records;
-
ordered history; and
-
join or association tables.
IcyDB will own allocation and define the behaviour precisely, including the permitted range and what happens at overflow. Applications will no longer need to build their own counter system for every table.
Why this order matters
The roadmap has three broad stages:
-
0.204–0.206: prove correctness, measure performance and remove the largest unnecessary query costs.
-
0.207–0.210: strengthen durability, persisted values, row evolution and large mutations.
-
0.211–0.213: add stronger constraints, integrity checking and generated identities.
These are not ten unrelated features. They are an ordered push to make IcyDB exact, efficient, durable and practical on the Internet Computer.
Or, put less formally: we now have a real SQL database running on the IC. I’m working through the remaining loose ends needed to make it extremely difficult to break.