# concepts The defaults work for most vaults. This page explains the three choices behind them: - the embedding model that turns text into vectors - the language model that answers - the shelf that holds a document Use it to decide when a default no longer fits. ``` python from tempfile import mkdtemp from vishalakshi import Vault from functools import partial from vishalakshi.ask import CHAT_CACHE, CachedChat, use_chat ``` ``` python v = Vault(Path(mkdtemp())/'vault.db') v.note('federate fuses the legs by rank because they share no vector space: the vault embeds ' 'prose, kosha embeds identifiers, ripgrep embeds nothing.', tags=['retrieval']) v.add('# Late chunking\n\n## Method\n\nEmbed the whole document, then pool per chunk, so a chunk ' 'keeps the context around it.\n\n## Results\n\nEvaluated on BEIR.', 'Late chunking', kind='note') with use_chat(partial(CachedChat, path=CHAT_CACHE)): v.connect() # replays a recorded extraction; plain `v.connect()` asks the model v.stats() ``` /Users/71293/code/personal/orgs/vishalakshi/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm {'docs': 2, 'nodes': 6, 'chunks': 3, 'encoder': 'model2vec', 'entities': 21, 'path': '/var/folders/kg/9vdw4mdd1fs58svgh4k1qhr09x7dqh/T/tmph5ysy_7x/vault.db', 'by_kind': {'note': 2}} ## What retrieval actually returns `v.context(q)` is the retrieval that `ask` uses. It returns whole sections. Each section has a breadcrumb and a `node_id` that `read` opens again. The first cell below prints the length and breadcrumb of each section in `results`. The code hits at the end come from kosha and ripgrep, because this repository is indexed. ``` python c = v.context('why are rankings fused instead of distances?', sections=4, related=4) for s in c.results: print(f'{len(s.text):5} {s.breadcrumb}') ``` 141 federate fuses the legs by rank because they share no vector space: the vault em 86 Late chunking › Method 18 Late chunking › Results 234 repo › /Users/71293/code/personal/orgs/vishalakshi/vishalakshi/code.py:57 154 grep › index.ipynb:161 600 repo › /Users/71293/code/personal/orgs/vishalakshi/vishalakshi/extract.py:179 89 grep › 07_concepts.ipynb:101 ``` python for s in c.related: print(f'{s.via:6} {s.breadcrumb}') ``` `results` are the sections that answer the question. `related` are the sections nearest to them in vector space. They give leads before you know what to search for. They do not change the ranking of `results`. The `via` field of each related section says how it was reached. ``` python c.encoder ``` 'minishlab/potion-multilingual-128M (256d, float16, model2vec)' `c.encoder` names the embedding model that produced the vectors. With `offline=True`, the vault hashes character sequences and uses no model. `evals/encoder.py` measures the cost of this against the default. ## Models and backends A model id names a model. [Rishi](https://github.com/vedicreader/rishi) works out which backend runs it. The local backends are LiteRT, MLX, llama.cpp and Ollama. A hosted API needs its API key. LiteRT is the default local backend. Rishi owns the backends and their options. They change with Rishi. This page shows only the two arguments that the vault passes on: `model=` and `chat_kw=`. ``` python import rishi.core from urai import Chat, infer_runtime from vishalakshi.ask import dflt_model # Rishi names and registers the backends. Urai resolves their model ids. gemma4_e2b = 'litert-community/gemma-4-E2B-it-litert-lm' dflt_model, gemma4_e2b # $VISHALAKSHI_MODEL if set, else the small local default ``` ``` python {m: infer_runtime(m) for m in (gemma4_e2b, 'mlx-community/Qwen3-4B-4bit', 'my-local.gguf', 'gpt-5.6-luna', 'gemma-3-4b-it-int4')} ``` ``` python v.ask('why are rankings fused?', model=gemma4_e2b) # an id: rishi knows what runs it v.ask('why are rankings fused?', model='llama/my-local.gguf') # a prefix when the id cannot say v.ask('why are rankings fused?', model='mlx-community/Qwen3-4B-4bit', chat_kw=dict(temp=0, think=True)) # the rest of rishi's constructor ``` ``` python from fastcore.test import test_fail # `model=` plus `chat_kw=` is the model API. Urai reports an id that names no registered backend. test_fail(lambda: Chat('gemma-3-4b-it-int4'), contains='backend') ``` A reasoning model writes its reasoning in a `` block before the answer. `ask` puts that block in `r.thinking` and the rest in `r.answer`. The block often mentions sections that the model then sets aside. [`cited`](https://vedicreader.github.io/vishalakshi/ask.html#cited) reads citations from the answer only. Retrieval never needs the network. Only answering with a hosted model does. ## Encoders An encoder, or embedding model, turns text into a vector. Texts with similar meanings get vectors that are close together. The default encoder is a static multilingual model. It produces 256 numbers per text, stored as 16-bit floats. `offline=True` hashes character sequences instead. This needs no download. It matches spelling and not meaning. A shelf records the encoder that wrote it. ``` python Vault(':memory:', offline=True).enc.note # never attempt a download; also the CI default ``` 'char-n-gram hashing (256d): lexical only; pass encoder= or restore network access for real semantics' ``` python Vault(encoder='minishlab/potion-science-32M') # pick a different one ``` Vault('/Users/71293/.vishalakshi/vault.db': 0 docs, 0 chunks, 0 entities, encoder=model2vec) 16-bit floats are the default width in litesearch. One vector index holds the vectors of one encoder. Documents embedded by different encoders need different shelves. `ENCODERS` maps three aliases to model ids:
alias use
multilingual The default.
code The model that kosha uses for identifiers.
gemma The slowest and most faithful.
Any other model2vec id also works by name. ``` python from vishalakshi.core import ENCODERS ENCODERS ``` A static model is a lookup table from words to vectors. It embeds a document in milliseconds without a GPU. Weighted mean reciprocal rank (MRR) measures how highly retrieval places relevant results. Higher scores are better. litesearch measured weighted MRR across four encoders (`evals/encoder.py`).
measure value
Encoders compared 4
Lowest measured score 0.018
Highest measured score 0.046
The static default trades some score for speed. `gemma` is an ONNX model. Choosing it loads onnxruntime. ``` python # `encoder=` also takes any object with an `.encode` method; `mk_encoder` wraps it for a shelf import numpy as np from vishalakshi.core import mk_encoder class Eight: def encode(self, xs, **kw): return np.zeros((len(xs), 8), dtype=np.float16) e = mk_encoder(Eight()) e.dims, e.method, e.name ``` One vector index is one vector space. Different encoders need different shelves. `SHELVES` names the shelves that the vault uses itself. `KIND_SHELF` maps each kind that acquisition adds to its own shelf. ``` python from vishalakshi.core import SHELVES, KIND_SHELF SHELVES, KIND_SHELF ``` ``` python papers = v.shelf('papers') # uses the live encoder unless a registry entry names another papers.add('# Late chunking\n\nWe evaluate contextual chunk embeddings on BEIR.', 'a paper') [(s['store'], s['encoder'], s['docs']) for s in v.shelves()] ``` `KIND_SHELF` routes documents as they arrive. `reshelf` moves a document later, when its doctype is known. `elsewhere` searches the other shelves. ``` python v.route('arxiv').name, v.route('web').name ``` ('papers', 'store') A PDF may be an invoice or a paper. Its kind does not say which. `categorize` decides after the document is filed. `reshelf` then moves the document to the shelf of its doctype. ``` python v.add('# Attention is all you need\n\n## Abstract\n\nWe propose the Transformer.\n\n' '## Introduction\n\nRelated work [1] et al.\n\n## References\n\ndoi:10.1/x', 'attention', source='/inbox/attention.pdf') r = v.reshelf('/inbox/attention.pdf', llm='never') r.doctype, r.was, r.store, r.moved ``` ('paper', 'store', 'papers', True) ``` python v.doc('/inbox/attention.pdf'), v.shelf('papers').doc('/inbox/attention.pdf')['title'] ``` (None, 'attention') Moving a document adds it to the new shelf from scratch. The encoder of the new shelf makes the vectors again. `reshelf` deletes the copy on the old shelf. The `meta` of the document moves with it. It includes the doctype and the shelf that the document came from. Marks do not move. A mark belongs to a document on its own shelf. `federate` runs a search on each shelf. It merges the results by rank, together with the kosha and ripgrep results. `elsewhere` adds a few sections from the neighbouring shelves to `context` and `ask`. ``` python v.federate('contextual chunk embeddings', repo=False, grep=False).legs ``` {'prose': 3, 'shelf:papers': 4} Each row from `elsewhere` has a `store` field that names its shelf. Pass `store` to `read(node_id, store=...)` when the section is not on the current shelf. ``` python e = v.elsewhere('contextual chunk embeddings') L(e).map(lambda r: (r.store, r.breadcrumb)) ``` [('papers', 'papers › a paper'), ('papers', 'papers › attention › Attention is all you need › Abstract')] ``` python v.read(e[0].node_id, store=e[0].store)['text'][:80] ``` '# Late chunking\n\nWe evaluate contextual chunk embeddings on BEIR.' A shelf records the encoder that wrote it and reopens with the same encoder. Opening a shelf with a different encoder raises an error, so two vector spaces never mix. To change the encoder of a shelf, drop the shelf and add its documents again.