đ https://github.com/RyanCodrai/turbovec
đ A vector index built on TurboQuant, written in Rust with Python bindings
ââââââââââââââââââââââââââââââ
What is turbovec?
is a fast, memoryâefficient vector search library written in Rust with Python bindings. It implements Google Researchâs TurboQuant algorithm â a dataâoblivious quantizer that needs no separate training phase and delivers nearâoptimal distortion.
Why youâll care
- A 10 Mâdocument floatâ32 corpus (~31 GB) fits in ~4 GB of RAM.
- Search is consistently faster than FAISS IndexPQFastScan (â3.4Ă speedâup at 4âbit, â20â30 % at 2âbit).
- No âtrainâthenâloadâ step â you can add vectors on the fly.
- Incremental, crashâsafe persistence (`sync`) writes only what changed.
- Builtâin filtering lets you restrict searches to an allowâlist without extra postâprocessing.
- Pureâlocal deployment â perfect for privacyâsensitive or latencyâcritical RAG pipelines.
Key features at a glance
- Online ingest: `add()` vectors anytime; no rebuilding.
- SIMDâoptimized search: handâwritten kernels (NEON SDOT/SMMLA, AVXâ512 VNNI, AVX2, scalar fallback).
- Incremental saves: `sync(path)` persists deltas with a single fsync; full snapshots still available via `write`/`load`.
- Filterâaware search: pass an id allowlist or slot bitmask; the kernel skips irrelevant blocks.
- Stable external IDs: `IdMapIndex` keeps your own uint64 identifiers and supports O(1) deletes.
- Framework adapters: dropâin replacements for LangChain, LlamaIndex, Haystack, Agno.
Getting started â Python
pip install turbovec
from turbovec import TurboQuantIndex
# create a 1536âdim index, 4âbit quantization
index = TurboQuantIndex(dim=1536, bit_width=4)
# add vectors (numpy float32, shape (n, dim))
index.add(vectors)
index.add(more_vectors)
# search
scores, ids = index.search(query, k=10)
# persistence
index.write("my_index.tv") # full snapshot
index.sync("my_index.tv") # incremental, crashâsafe
loaded = TurboQuantIndex.load("my_index.tv")
Stable IDs example
from turbovec import IdMapIndex
import numpy as np
idx = IdMapIndex(dim=1536, bit_width=4)
idx.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))
scores, external_ids = idx.search(query, k=10)
idx.remove(1002) # O(1) delete by id
idx.sync("my_index.tvim")
Hybrid (filtered) search â combine a coarse external retriever with dense reranking:
allowed = np.array(db.execute(
"SELECT id FROM docs WHERE tenant=?", (t,)
).fetchall(), dtype=np.uint64)
scores, ids = idx.search(query, k=10, allowlist=allowed)
The filter is evaluated inside the SIMD kernel, so only the allowed blocks incur any computation.
Getting started â Rust
cargo add turbovec
use turbovec::TurboQuantIndex;
let mut index = TurboQuantIndex::new(1536, 4).unwrap();
index.add(&vectors);
let (scores, ids) = index.search(&queries, 10);
index.write("index.tv").unwrap();
let loaded = TurboQuantIndex::load("index.tv").unwrap();
(1/2)
