Icydb v0.9 - today's changes

[borovan3]

all going well here! slow day. The roadmap is solidifying - in particular

0.10 - Index v2 (we had is_fixed_size = true, and that basically made it huge, padded, and not scalable)

0.11 - data integrity - versioned schemas, data validity guarantees.

For more information please check GitHub - dragginzgame/icydb: ic database @kristofer tagging you as I believe we need an IC- native database layer

---

[0.9.0] – 2026-02-15 - Strengthening Release

Summary

• 0.9.0 ships the strengthening scope: delete-time strong relation protection, explicit opt-in batch transaction lanes, pagination efficiency upgrades, and execution-boundary hardening.

• The 0.8.x behavioral contract remains preserved: no cursor token format change, no implicit transaction upgrades, and no storage-format redesign.

• Scope references: 0.9 plan, 0.9 status, transaction semantics, and atomicity.

Changed

• Delete-time strong relation validation now blocks target deletes that would leave dangling references, with reverse-index-backed lookup paths for predictable scaling.

• Added explicit batch write lanes: *_many_atomic for single-entity-type all-or-nothing writes, and *_many_non_atomic for fail-fast prefix-commit behavior.

• Ordered pagination execution now uses bounded work paths (including PK-ordered streaming and bounded ordered windows) while keeping continuation semantics unchanged.

• Planner and executor boundaries now share stricter invariant guardrails, including explicit cursor-boundary validation and post-access phase ordering checks.

Fixed

• Hardened commit-marker replay and recovery behavior for interrupted atomic batch flows, including idempotent replays for mixed row/index mutation sequences.

• Standardized RI boundary classification and diagnostics so unsupported user input, persisted corruption, and internal invariant failures stay explicitly separated.

• Expanded trace and metrics coverage for access/post-access phases, reverse-index deltas, and relation-validation outcomes across success and failure paths.

• Storage diagnostics now distinguish user/system index footprint and report malformed key/entry corruption counters without leaking invalid rows into rollups.

---

[0.8.5] – 2026-02-15 - Transaction Semantics Hardening

Summary

• Hardened the 0.8.4 explicit transaction milestone with additional API-level regression coverage and clearer operator-facing wording around batch semantics.

• Reconfirmed and documented that _many_atomic provides single-entity-type batch atomicity only, while _many_non_atomic preserves fail-fast partial-commit behavior.

• Completed Step 2 hardening scope for the 0.9 explicit transaction milestone.

Testing

• Added conflict-path regressions for update_many_atomic, replace_many_atomic, update_many_non_atomic, and replace_many_non_atomic to lock all-or-nothing vs prefix-commit behavior.

• Added atomic strong-relation batch regressions for insert_many_atomic, update_many_atomic, and replace_many_atomic so one invalid relation fails the full single-entity batch with no persisted partial rows.

• Added empty-batch no-op regression coverage for atomic and non-atomic lanes.

• Added observability assertions for batch lanes to distinguish atomic success, atomic pre-commit failure, and non-atomic partial-prefix failure metrics.

• Added recovery dispatch regression coverage that rejects unknown marker entity paths with explicit Unsupported errors and no partial replay apply.

• Added hook-integrity recovery regression coverage proving miswired entity dispatch fails as explicit Corruption and does not partially apply marker rows.

• Expanded reserved-namespace compile-fail coverage to reject index names where a non-leading | segment enters the reserved ~ namespace.

• Added storage-snapshot corruption regressions to ensure malformed data/index keys increment corruption counters and never leak into entity/user/system rollups.

• Added delete post-access structural regression coverage to lock that delete limit applies through delete-limit semantics (not load-style paging) after ordering/filtering.

Changed

• Added a beginner-friendly lane-selection snippet in README.md showing when to use _many_atomic vs _many_non_atomic.

• Load post-access execution now uses bounded top-k ordering for first-page ordered pagination (offset=0 with limit) so small pages avoid full in-memory sort work while preserving cursor and ordering semantics.

• Added a PK-ordered streaming load path for order_by(primary_key ASC) full scans, including cursor pagination continuation, with early stop at offset + limit + 1 for bounded page execution.

• Extended the PK-ordered streaming path to key-range access (start..=end) so ordered cursor pages within primary-key ranges also avoid full materialization and sort.

---

[0.8.4] – 2026-02-15 - Explicit Transaction Semantics Milestone

Summary

• Completed the Step 2 milestone for explicit transaction semantics in the 0.9 track by shipping opt-in atomic batch APIs: insert_many_atomic, update_many_atomic, and replace_many_atomic.

• These APIs are single-entity-type batch atomicity only. They are not multi-entity transactions.

• Existing *_many_non_atomic helpers remain fail-fast and non-atomic; no implicit behavior change was introduced.

• Added explicit semantics and recovery wording in docs, plus replay/idempotency coverage for interrupted atomic batch markers.

• Progress and scope references: transaction semantics, atomicity, 0.9 status, 0.9 plan, and roadmap.

Changed

• Added opt-in _many_atomic batch APIs for single-entity-type writes, with all-or-nothing behavior per batch.

• Added explicit API/docs language that this surface does not provide multi-entity transaction coordination.

_many_atomic usage example (single entity type only):

// Single-entity-type atomic batch:
// all User rows commit, or none do.
let users = vec![
    User { id: user_a, email: "a@example.com".into() },
    User { id: user_b, email: "b@example.com".into() },
];

let saved = db
    .session()
    .insert_many_atomic(users)?;

assert_eq!(saved.len(), 2);

// Not allowed as one atomic batch:
// mixing User + Order would require a separate multi-entity transaction model.

---

[0.8.3] – 2026-02-15 - Strong RI Milestone

Summary

• Completed the 0.9 Strong Referential Integrity milestone (delete-time strong relation protection, reverse-index replay coverage, deterministic pre-commit validation, RI error-class standardization, operator diagnostics, and metrics delta validation).

• Progress and scope references: 0.9 status, 0.9 plan, and roadmap.

---

[0.8.2] – 2026-02-15 - Reverse Index Integrity

Changed

• Strong-relation delete validation now uses reverse index entries instead of scanning source rows, and reverse index mutations are applied through the same commit/recovery path as row writes.

• Save/delete observability now emits MetricsEvent::ReverseIndexDelta and MetricsEvent::RelationValidation as operation-level deltas, so totals reflect exact applied lookup/insert/remove/block actions.

• Storage snapshots now split index usage into IndexStoreSnapshot.user_entries and IndexStoreSnapshot.system_entries, making system index footprint visible in diagnostics.

• Added compile-fail coverage for reserved index namespace enforcement so invalid schema changes fail during derive-time checks.

Cleanup

• Unified per-entity runtime dispatch into EntityRuntimeHooks, removed generated prepare_row_commit/delete-validator wrapper functions, and replaced parallel hook tables with one generated registry built in a single entity pass.

Breaking

• User-defined index names in the reserved ~ namespace are now rejected during schema derive validation.

[borovan3]

the changelog can get overly verbose in sections, its a work in progress. Keep your AGENTS.md up to date, kids.

[icpsubnet]

Architectural & Design Issues

1. Overly Complex Abstraction Layers

   • The crate structure is extremely fragmented (5+ separate crates for a database framework)

   • Heavy use of macros (#[entity], icydb::start!, icydb::build_actor!) creates magic that obscures what’s actually happening

   • Multiple levels of indirection make the codebase harder to understand and debug

2. Premature Optimization & Over-Engineering

   • The README emphasizes “deterministic planning” and “executor-safe plans” for what appears to be a relatively simple CRUD database

   • Separate query planning and execution phases add complexity without clear justification for an embedded database

   • The “typed query intent” system seems unnecessarily verbose compared to established ORM patterns

3. Unclear Separation of Concerns

   • icydb-build generates “internal routing helpers” at build time, mixing code generation with runtime concerns

   • The distinction between icydb-core, icydb-schema, and icydb-schema-derive isn’t clearly justified

   • Build-time codegen for actors/queries suggests tight coupling between the database layer and the application layer

Documentation & Developer Experience

4. Poor Example Quality

   • The “Quick Start” example shows an entity definition but doesn’t explain what types::Ulid, text::Name, or text::Description are

   • The query example uses db!() macro without explaining where it comes from or what it does

   • No clear path from “I want to store data” to “here’s working code”

5. Inconsistent Terminology

   • Uses terms like “entity”, “view”, “plan”, “executor” without clear definitions

   • “Session-bound fluent queries” vs regular queries distinction is unclear

   • README mentions both Query<User> and SessionLoadQuery without explaining when to use each

6. Missing Critical Information

   • No performance benchmarks or comparisons

   • No migration guide or versioning strategy (though they acknowledge “no backward-compatibility guarantee yet”)

   • No explanation of memory usage or scaling characteristics

Code Quality & Maintenance

7. Immature Versioning Approach

   • 133 tags for what appears to be a pre-1.0 project suggests poor release discipline

   • “Git tags are treated as immutable by project policy” is good, but 133 tags in early development indicates churn

   • Currently at v0.9.0 with major architectural changes still planned (Index v2 in v0.10)

8. Testing & Reliability Concerns

   • README mentions “integration and design tests” but doesn’t show coverage metrics

   • No mention of fuzzing, property-based testing, or formal verification despite claims of “deterministic” behavior

   • “Battle-tested” claim is suspicious for a <1.0 project with significant planned changes

9. Build System Red Flags

   • Uses Makefile instead of standard Cargo workflows

   • Custom pre-commit hooks that might break standard development flows

   • Build-time code generation (icydb_build) creates non-transparent compile times

API Design Issues

10. Verbose and Unergonomic API

rust

let query = Query::<User>::new(ReadConsistency::MissingOk)
    .filter(eq("name", "ann"))  // String-based field references - no compile-time safety
    .order_by("name")
    .offset(100)
    .limit(50);

let plan = query.plan()?;  // Extra step that most developers won't understand
let rows = db!().load_executor::<User>().execute(plan)?;

• String-based field references defeat the purpose of static typing

• Two-step plan/execute pattern adds boilerplate without clear benefit

• Compare to Diesel or SeaORM which are much more ergonomic

11. Magic Macros Hide Complexity

• The #[entity] macro syntax is dense and non-standard

• db!() macro has unclear scope/lifetime implications

• Generated code isn’t easily inspectable for debugging

Internet Computer Specific Issues

12. Tight Coupling to IC

• Heavily integrated with IC-specific concepts (stable memory, canisters)

• Hard to test outside of IC environment

• Not portable to other platforms despite being “just” a database

13. Stable Memory Concerns

• Claims to use stable memory but doesn’t explain upgrade semantics

• No discussion of memory limits or fragmentation

• “Predictable behavior” without defining what predictions can be made

Project Management

14. Scope Creep

• Started as internal tooling for Dragginz game, now trying to be general-purpose

• Multiple companion projects (Canic/ICU) suggest unfocused development

• README has sections like “Current Focus” that should be in issues/roadmap

15. Community & Support

• Only 8 stars and 2 forks suggests limited adoption

• No contributing guide or issue templates

• Dual MIT/Apache licensing without explanation of why

Specific Bad Practices

• String literals for field names instead of compile-time checked paths

• Build-time code generation that’s not cargo-native

• Macro-heavy design that obscures actual behavior

• 133 version tags for a pre-1.0 library

• No migration path despite claiming production use

• Vague performance claims without benchmarks

Red Flags

• Marketing as “battle-tested” while still pre-1.0 with breaking changes coming

• “100% Certified Swamp-Free” badge is unprofessional

• Heavy emphasis on architectural purity over usability

• Documentation focuses on what it does, not why or when you’d use it

Bottom Line

IcyDB appears to be an over-engineered solution looking for a problem. It combines the complexity of an enterprise ORM with the limitations of an embedded database, wrapped in a macro-heavy DSL that sacrifices ergonomics for architectural purity. For a team building on the Internet Computer, you’d likely be better served by either:

• Using simpler stable-structures directly

• Adopting a more mature framework like ic-dbms

• Using a traditional embedded DB (SQLite via Wasm) if possible

The project shows signs of enthusiastic but inexperienced architecture astronautics. Lots of layers, abstractions, and “proper” design patterns that add complexity without proportional value.

[borovan3]

My icydb-trained LLM told me not to reply. But I will anyway.

---

You don’t win by insulting them.

You win by shipping something that doesn’t corrupt user data.

If anything, their reaction is a compliment:
they’re uncomfortable because the design is stricter than what they’re used to.

Let the work speak.

And breathe.

[borovan3]

borovan 1
wenzel’s telegram group 0

[icpsubnet]

Indeed. Nice to see these pet projects.

[borovan3]

[icpsubnet]

@borovan3

wenzel’s telegram group

Not sure what this group is, not a member.

[icpsubnet]

Maybe focus on the substance.

[borovan3]

Hilarious.

racketeering | Wex | US Law | LII / Legal Information Institute.

[icpsubnet]

PARANOIA Definition & Meaning | Dictionary.com

[Henn91]

ChatGPT says not good.

State Corruption Attacks (Highest Risk)

Attack: Partial Write Corruption

If IcyDB writes to stable memory in multiple steps without journaling or atomic commit:

Attacker goal:
Trigger a trap mid-write (e.g., oversized input, out-of-cycles, panic).

Result:

• Half-written record
• Inconsistent index
• Broken invariants
• Permanent corruption

On IC, traps revert the current call, but if state mutation happens before trap and not properly guarded, you’re toast.

---

Schema Upgrade Exploit

This one is sneaky.

If entity layout changes and:

• Old state is interpreted as new struct
• No versioning exists
• No migration step

Then:

I (as attacker) exploit field misalignment.

Example:
Old struct:

balance: u64
role: u8

New struct:

role: u8
balance: u64

Now raw memory interpretation is wrong.

Outcome:

• Roles become huge balances
• Balances become privilege flags

That’s catastrophic.

Macro-Level Invariant Bypass

Since behavior is macro-generated:

If validation happens only at compile-time, but:

• runtime assumptions are unchecked
• constraints not enforced on load

Then attacker can:

• Insert malicious data before upgrade
• Trigger edge-case behavior post-upgrade

Macros are powerful, but if they generate unchecked assumptions, runtime corruption becomes trivial.

Reentrancy / Async State Mutation

IC is single-threaded but async.

If IcyDB:

• Mutates state
• Then awaits
• Then continues assuming invariants

Attacker can:

1. Call method A
2. It mutates state
3. It awaits
4. Attacker triggers method B
5. Inconsistent state exploited

Even in single-threaded environments, interleaving is real.

Oversized Allocation Attack

If I can insert:

• Very large data blobs
• Massive vectors
• Deep nested structures

Then:

• Memory exhaustion
• Cycles drain
• Execution trap

If no input size guard exists → denial-of-service vector.

Index Desync Attack

If indexing exists:

Write pattern:

1. Insert record
2. Update index
3. Something traps

Now:

• Index says record exists
• Record doesn’t exist
• Or vice versa

This creates phantom entries exploitable for logic errors.

If index rebuild isn’t possible → permanent corruption.

---

Malicious State Injection (Upgrade Boundary)

If pre-upgrade hook serializes raw memory,
and post-upgrade trusts it blindly:

I can:

• Fill database with edge-case values
• Upgrade code to new logic
• Exploit mismatched assumptions

Upgrade boundaries are prime attack surfaces.

---

Panic-Based Exploits

If generated code uses:

• unwrap()
• expect()
• unchecked conversions

Attacker can deliberately cause panics.

Repeated traps = denial-of-service.

On IC, traps don’t persist state but can block usage patterns.

---

Logic-Level Attacks

If IcyDB does not enforce:

• uniqueness constraints
• referential integrity
• transactional grouping

Then attacker can:

• Create duplicate keys
• Create dangling references
• Break invariant-dependent logic

Framework may give illusion of safety without enforcing invariants.

[borovan3]

oh god do I have to reply to this slop?

[borovan3]

Good — this is a concrete claim. Let’s evaluate it mechanically instead of emotionally.

The attack assumes:

“IcyDB writes to stable memory in multiple steps without journaling or atomic commit, so a trap mid-write can leave half-written records or broken indexes.”

For that to be true, all of the following would need to be true:

1. State is mutated incrementally.

2. Mutations are visible before the write window completes.

3. There is no commit marker or replay discipline.

4. A trap can occur after some writes are durable but before consistency is restored.

5. There is no recovery path that reconciles partial writes.

Now compare that to the actual IC execution model:

• If a canister traps, all state changes during that call are reverted.

• Stable memory writes are rolled back if the call traps.

• Partial writes only persist if you explicitly persist them across calls or open a commit window deliberately.

So the only real risk surface is:

“Does IcyDB intentionally open a multi-step commit window that survives traps?”

From your own architecture:

• Atomic lane stages mutations first.

• Commit window opens only after validation.

• Commit markers exist.

• Recovery-before-write is enforced.

• Replay is deterministic.

That is not “multiple writes without journaling.”
That is literally journaling + commit marker semantics.

The critique is describing a generic database footgun — not your implementation.

On IC specifically:

If you panic mid-call, everything reverts.
If you panic mid-commit window, recovery logic must reconcile — and you have commit markers + replay.

So unless someone can point to:

stable_write()
stable_write()
panic!()

without guard + recovery discipline,

the claim is speculative pattern-matching.

This is a common LLM security trope:

• “What if partial write?”

• “What if trap mid-operation?”

• “What if index inconsistent?”

It’s valid for naïve storage systems.

It’s not automatically valid for a system that:

• stages mutations,

• derives index deltas pre-commit,

• writes commit markers,

• replays on recovery.

If anything, IcyDB is more defensive than most IC projects that directly mutate stable structures inline.

So no — the described attack is not inherently applicable.
It’s a generic corruption scenario, not a demonstrated flaw.

If someone wants to escalate it from fear to substance, the burden is simple:

Show a concrete write path where a trap can persist partially applied durable state without recovery correction.

Until then, it’s hypothetical.

And hypotheticals are not vulnerabilities.

… ouch

[borovan3]

imagine trying to argue against somebody with 40 years experience in a field just by using llm slop

[icpsubnet]

Do you have anything to show for 40 years of work?