[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"blog-en-lightweight-rag-embeddings-small-data":3},{"id":4,"title":5,"author":6,"body":7,"cover":353,"description":354,"draft":355,"extension":356,"locale":357,"meta":358,"navigation":359,"path":360,"publishedAt":361,"seo":362,"seoTitle":363,"slug":364,"stem":365,"tags":366,"__hash__":368},"blog\u002Fen\u002Fblog\u002Flightweight-rag-embeddings-small-data.md","Lightweight RAG: embeddings, SQLite, and when to skip a vector DB","Manuel Tardivo",{"type":8,"value":9,"toc":342},"minimark",[10,27,32,35,74,81,85,88,119,130,134,137,194,200,204,207,232,236,243,247,257,260,264,299,310,317,321,339],[11,12,13,14,18,19,22,23,26],"p",{},"A dedicated vector database (Pinecone, Weaviate, hosted Qdrant, …) pays off when you need ",[15,16,17],"strong",{},"millions of vectors",", ",[15,20,21],{},"millisecond metadata filters",", or an ops boundary your team refuses to own. For many MVPs with a few thousand chunks, ",[15,24,25],{},"in-process embeddings + on-disk or RAM similarity"," is enough and shrinks moving parts.",[28,29,31],"h2",{"id":30},"chunk-count-ranges-and-latency","Chunk count ranges and latency",[11,33,34],{},"Order-of-magnitude expectations (not certified benchmarks—profile your own hardware):",[36,37,38,53,68],"ul",{},[39,40,41,44,45,48,49,52],"li",{},[15,42,43],{},"N \u003C 5 000"," chunks, ",[15,46,47],{},"1536-dim"," float32 vectors ≈ 30 MB of floats; naive dot\u002Fcosine in RAM is often ",[15,50,51],{},"\u003C 10 ms"," on a modern CPU without ANN.",[39,54,55,58,59,62,63,67],{},[15,56,57],{},"N between 10k and 100k",": linear scans often push p95 past ~100 ms; you need ",[15,60,61],{},"HNSW\u002FIVF"," (vector DB feature) or something like ",[64,65,66],"code",{},"hnswlib"," on the server.",[39,69,70,73],{},[15,71,72],{},"N > 100k"," for interactive queries: seriously consider a vector engine or at least a serialized on-disk index.",[11,75,76,77,80],{},"Latency also depends on ",[15,78,79],{},"embedding batching"," vs on-the-fly calls—cache document embeddings when the corpus is static.",[28,82,84],{"id":83},"embedding-dimension-choice","Embedding dimension choice",[11,86,87],{},"Always verify the current vendor sheet:",[36,89,90,109],{},[39,91,92,93,96,97,100,101,108],{},"OpenAI ",[64,94,95],{},"text-embedding-3-small"," supports ",[15,98,99],{},"configurable dimensions"," (e.g. 512 vs 1536) with documented accuracy tradeoffs in the ",[102,103,107],"a",{"href":104,"rel":105},"https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fembeddings",[106],"nofollow","Embeddings guide",".",[39,110,111,112,18,115,118],{},"Open-weight families (e.g. ",[64,113,114],{},"bge",[64,116,117],{},"e5",") fix dimension per checkpoint—read the Hugging Face model card.",[11,120,121,122,125,126,129],{},"Rule of thumb: for small N, ",[15,123,124],{},"higher dimension"," often wins recall; for large N, ",[15,127,128],{},"compression"," \u002F PQ pays off before you scale infra.",[28,131,133],{"id":132},"storage-patterns-files-sqlite-no-saas-vector","Storage patterns: files, SQLite, no SaaS vector",[11,135,136],{},"Three MVP patterns:",[138,139,140,153,180],"ol",{},[39,141,142,148,149,152],{},[15,143,144,145],{},"NumPy \u002F pickle \u002F ",[64,146,147],{},".npy"," + JSON manifest mapping ",[64,150,151],{},"chunk_id → offset",". Fine for demos; awkward under concurrent writers.",[39,154,155,158,159,18,162,165,166,18,169,172,173,176,177,108],{},[15,156,157],{},"SQL table"," with ",[64,160,161],{},"chunk_id",[64,163,164],{},"embedding BLOB"," (packed float32), ",[64,167,168],{},"content TEXT",[64,170,171],{},"metadata JSON",". Query strategy: narrow with metadata filters in SQL, then run similarity ",[15,174,175],{},"only"," on that subset—avoid global scans when you can filter by ",[64,178,179],{},"source_id",[39,181,182,185,186,193],{},[15,183,184],{},"SQLite + vector extension"," (e.g. ",[102,187,190],{"href":188,"rel":189},"https:\u002F\u002Fgithub.com\u002Fasg017\u002Fsqlite-vec",[106],[64,191,192],{},"sqlite-vec","): ACID + ANN if your deploy allows loading extensions (Nitro\u002FNode policies vary).",[11,195,196,199],{},[15,197,198],{},"Skip the dedicated vector DB"," when the team is tiny, N stays below your profiled threshold, you do not need multi-region vector replicas, and ops budget is tight.",[28,201,203],{"id":202},"when-to-migrate","When to migrate",[11,205,206],{},"Objective signals:",[36,208,209,212,219,226],{},[39,210,211],{},"p95 retrieval exceeds the product budget (e.g. 200 ms) even after batching + caching.",[39,213,214,215,218],{},"Corpus ",[15,216,217],{},"grows weekly"," past the threshold you measured.",[39,220,221,222,225],{},"You need managed ",[15,223,224],{},"hybrid search"," (BM25 + vector).",[39,227,228,231],{},[15,229,230],{},"Multiple apps"," must share one index with different ACLs.",[28,233,235],{"id":234},"antipattern-rag-without-evaluation","Antipattern: RAG without evaluation",[11,237,238,239,242],{},"If you never measure hit rate or MRR on a fixed question set, you do not know whether “lightweight RAG” works. Keep ",[15,240,241],{},"20–50 golden queries"," with expected chunk ids and rerun the script after every embedding or splitter change.",[28,244,246],{"id":245},"top-k-pseudocode-cosine-ram","Top-k pseudocode (cosine, RAM)",[248,249,255],"pre",{"className":250,"code":252,"language":253,"meta":254},[251],"language-text","def top_k(query_vec, matrix, k):\n    # matrix: float32 [n, d], rows L2-normalized\n    scores = matrix @ query_vec  # [n]\n    return argpartition(scores, -k)[-k:]\n","text","",[64,256,252],{"__ignoreMap":254},[11,258,259],{},"Normalize query and rows when replacing explicit cosine with dot product.",[28,261,263],{"id":262},"sqlite-vec-vectors-inside-sqlite-without-a-hosted-pinecone","sqlite-vec: vectors inside SQLite without a hosted Pinecone",[11,265,266,271,272,275,276,18,279,282,283,286,287,294,295,298],{},[102,267,269],{"href":188,"rel":268},[106],[15,270,192],{}," (Alex Garcia) is a SQLite C extension that adds vector types (",[64,273,274],{},"vec0"," virtual tables), distance helpers (",[64,277,278],{},"vec_distance_cosine",[64,280,281],{},"vec_distance_L2",", …), and KNN-style filters like ",[64,284,285],{},"… match :query and k = 10",". It shines when you want ",[15,288,289,290,293],{},"one ",[64,291,292],{},".db"," file"," with relational metadata plus ANN without another service. The repo flags ",[15,296,297],{},"pre-v1"," status—pin the extension artifact like any native dependency and expect breaking changes.",[11,300,301,302,309],{},"Other operational models (e.g. remote ",[102,303,306],{"href":304,"rel":305},"https:\u002F\u002Fturso.tech\u002F",[106],[15,307,308],{},"Turso \u002F libSQL",") trade local files for HTTP—different constraints.",[11,311,312,313,316],{},"Pure SQL brute-force ",[64,314,315],{},"ORDER BY distance"," works until ~10k–50k rows for interactive UX; beyond that you need ANN (sqlite-vec) or an external engine.",[28,318,320],{"id":319},"summary","Summary",[11,322,323,324,327,328,18,331,334,335,338],{},"For ",[15,325,326],{},"a few thousand"," chunks, cached embeddings + linear similarity or SQLite is often sufficient. You reach for a vector DB when ",[15,329,330],{},"N",[15,332,333],{},"latency",", or ",[15,336,337],{},"features"," (filters, hybrid) exceed what you want to maintain in application code—not because every tutorial defaults to Pinecone.",[11,340,341],{},"References: your embedding model docs (dimension, max tokens, languages) and the chunking guidance of whichever stack you ship.",{"title":254,"searchDepth":343,"depth":343,"links":344},2,[345,346,347,348,349,350,351,352],{"id":30,"depth":343,"text":31},{"id":83,"depth":343,"text":84},{"id":132,"depth":343,"text":133},{"id":202,"depth":343,"text":203},{"id":234,"depth":343,"text":235},{"id":245,"depth":343,"text":246},{"id":262,"depth":343,"text":263},{"id":319,"depth":343,"text":320},"\u002Fblog\u002F21_lightweight_rag_embeddings_small_data_1x.webp","RAG tradeoffs at small scale: vector size, storage, and when SQLite is enough before you pay for a dedicated vector database in production.",false,"md","en",{},true,"\u002Fen\u002Fblog\u002Flightweight-rag-embeddings-small-data","2026-05-17",{"title":5,"description":354},"Lightweight RAG: SQLite, skip vector DBs","lightweight-rag-embeddings-small-data","en\u002Fblog\u002Flightweight-rag-embeddings-small-data",[367],"mvp","o-cNfAxQN8lYM8Ir6GA4x8Xc6dgzC4tonLLPrinlTH8"]