New Libraries ic-rig + ic-hnsw — LLM apps and vector search for the Internet Computer

Developers Rust
1

Hey all :waving_hand:

Shipped two crates this week that pair well if you’re building anything RAG/embeddings-shaped on ICP, so figured I’d post them together rather than spam two threads.

ic-rig — bring-your-own-HTTP-client LLM library

The problem: most Rust LLM libraries assume reqwest + tokio, which doesn’t fit a canister’s HTTP outcall model. ic-rig is generic over a simple HttpClient trait instead — swap in ic_cdk outcalls, reqwest, or a mock for tests, and the provider logic doesn’t care.

  • Four providers behind one CompletionModel trait: OpenAI, Anthropic, Gemini, DeepSeek — feature-gated, compile only what you use.
  • Built-in agentic loop that dispatches tool calls and feeds results back until the model returns a final answer.
  • Embeddings support, plus a lightweight LSH vector store with a few configurable distance metrics for basic in-memory semantic search.

:package: https://crates.io/crates/ic-rig
:laptop: https://github.com/Zedonboy/ic-rig

ic-hnsw — persistent, canister-native ANN search

The problem ic-rig’s LSH store doesn’t try to solve: real, scalable approximate nearest-neighbour search that survives upgrades. ic-hnsw brings proper HNSW indexing to canisters:

  • Built on ic-stable-structures, so the whole graph + vectors persist across upgrades with no extra plumbing.
  • Deploy standalone and call it from anywhere, or embed it as a library directly in your own canister.
  • Collections, multiple distance metrics, tunable HNSW params (M, ef) to trade recall against instruction budget.
  • Access control built in, including a controller-only lock for sealing a canister post-setup.

:package: https://crates.io/crates/ic-hnsw
:laptop: https://github.com/Zedonboy/ic-hnsw

Where they meet

They’re independent crates, not wired together in code, but the shapes fit: generate/call out to a model and produce embeddings with ic-rig, then hand those vectors to ic-hnsw when you need real persistent ANN search instead of ic-rig’s in-memory LSH store — e.g. a RAG pipeline where the corpus is too big or long-lived for LSH to make sense.

Both are early (v0.1.0) — feedback, issues, “here’s what I’m building” stories all very welcome on either repo.

4 Likes