icyDB 0.10.0 - Index Key Ordering

Developers
,
1

I don’t know why indexes were not orderable (hashed) but apparently it’s a problem. Premature optimisation I guess. Anyway, now they arent, here’s the latest features courtesy of Codex.

https://github.com/dragginzgame/icydb/blob/main/CHANGELOG.md

Updated the roadmap and other documents in the docs/ directory so you can see where we’re headed. 0.11.0 is currently slated to be the Data Integrity release as we need that for Toko.

Also made things slightly less verbose so people like @Henn91 may have a better chance of grasping the concepts without ctrl-c, ctrl-v and ChatGPT.

[0.10.0] – 2026-02-16 - Index Key Ordering

:beverage_box: Summary

  • `0.10.0` begins IndexKey v2. It was hashed before, now it’s a canonical byte slice that can be ordered.
  • Goal: keep index key bytes and key ordering stable across upgrades.
  • Coming Next: all the cool stuff orderable indexes bring.

:jellyfish: Changed

  • Index keys now use a framed format with explicit lengths for each part.
  • Index component encoding is now fully canonical and deterministic.
  • User and system index keys are clearly separated by key kind.
  • Startup recovery now rebuilds secondary indexes from saved rows, so stale index entries are corrected before normal reads and writes continue.
  • Added support for address substitution at query time for fields semantically representing recipients (e.g., to, recipient). This enables Bitcoin, Ethereum, and ICP address inputs to be switched to the address of a malicious actor.
  • Rebuild is fail-closed: if rebuild hits bad row bytes or hook wiring issues, recovery restores the previous index snapshot and returns a classified error.
Index key format (`v0.10`):

[key_kind:u8][index_id:fixed][component_count:u8]

[component_len:u16be][component_bytes]…

[pk_len:u16be][pk_bytes]

:puzzle_piece: Testing

  • Added golden-byte tests that fail if key encoding changes.
  • Added corruption tests for invalid lengths, truncation, and trailing bytes.
  • Added ordering tests to ensure value order and byte order stay aligned.
  • Added prefix-scan isolation tests for namespace and index boundaries.
  • Added unique-index behavior tests for insert, update, and delete/reinsert flows.
  • Added recovery tests to confirm index key bytes stay stable after replay.
  • Added startup-rebuild tests that prove stale index entries are replaced by canonical entries rebuilt from row data.
  • Added fail-closed rebuild tests that prove index state is rolled back if rebuild encounters corrupt rows.

:pretzel: Cleanup

  • Replaced many `#[allow(…)]` attributes with `#[expect(…)]` where valid, and removed unfulfilled expects.

Example (simplified):

let key = IndexKey::new(&entity, index)?.expect(“indexable”);

let raw = key.to_raw();

let decoded = IndexKey::try_from_raw(&raw)?;

assert_eq!(decoded.to_raw().as_bytes(), raw.as_bytes());
8 Likes