answer from the vault, with citations back into it
rishi picks its backend from the shape of a model id — litert-community/… or .litertlm goes to LiteRT, .gguf to llama.cpp, mlx-community/… to MLX, a hosted name like claude-sonnet-5 to fastllm — and raises rather than guessing when the id says nothing. MODELS exists only so the common cases are one short word.
(answer, thinking) from a reasoning model’s reply, whichever half of the tag pair arrived.
rishi’s split_think covers <think>…</think> and an opener cut off at the token cap. MLX chat templates prefill the opener, so a Qwen reply arrives with a closing</think> and nothing before it — which split_think leaves alone and cited would then read for citations the answer never makes.
def mk_prompt( question:str, # what you want to know ctx, # AttrDict from Vault.context() max_chars:int=4000, # chars kept per section related:bool=True, # include the associative leg)->str:
The user turn: the numbered sections, then the question.
The numbering is the contract with the model: [n] in the answer maps to ctx.results[n-1], which is what makes an answer checkable against the vault instead of merely plausible.
def ask( question:str, # what you want to know model:str=None, # a MODELS alias, a full id, or None -> $VISHALAKSHI_MODEL runtime:str=None, # 'litert' | 'mlx' | 'llama' | 'remote'; inferred from the id if None sections:int=6, # operative sections retrieved related:int=6, # associative sections offered as leads kind:str=None, # restrict retrieval to one or more KINDS max_chars:int=4000, # chars of each section shown to the model sp:str="You answer questions from a personal research vault.\n\nYou are given numbered sections retrieved from the user's own corpus — papers, web pages,\ntranscripts, files and their own notes. Answer only from those sections.\n\nRules:\n- Cite every claim with the bracketed number of the section it came from, like [2]. A sentence\n drawing on two sections cites both.\n- If the sections do not answer the question, say exactly what is missing rather than filling the\n gap from memory. A vault that admits a hole is useful; one that guesses is not.\n- Sections marked RELATED were reached by association, not by matching the question. Use them for\n context or to point somewhere worth reading next, and say so when you do.\n- Prefer the user's own notes when they conflict with a source, and flag the disagreement.", # system prompt fresh:bool=True, # start a new conversation rather than continuing the last**kw)->AttrDict: # forwarded to Vault.context
Retrieve, then answer with citations back into the vault.
cited resolves the [n] markers in the answer back to node_ids you can read(), so every claim is one call away from the text it came from; context is the full retrieval, kept so you can inspect what the model was and was not shown. A reasoning model’s <think> block is split off into thinking rather than left in answer: it names sections it then discards, so leaving it in would put citations in the answer that the answer does not actually make.
def chat( model:str=None, runtime:str=None, sp:str="You answer questions from a personal research vault.\n\nYou are given numbered sections retrieved from the user's own corpus — papers, web pages,\ntranscripts, files and their own notes. Answer only from those sections.\n\nRules:\n- Cite every claim with the bracketed number of the section it came from, like [2]. A sentence\n drawing on two sections cites both.\n- If the sections do not answer the question, say exactly what is missing rather than filling the\n gap from memory. A vault that admits a hole is useful; one that guesses is not.\n- Sections marked RELATED were reached by association, not by matching the question. Use them for\n context or to point somewhere worth reading next, and say so when you do.\n- Prefer the user's own notes when they conflict with a source, and flag the disagreement.",**kw):
A rishi.Chat bound to this vault’s system prompt, cached on the vault.
Four backends, one call: litert (CPU, no API key, the default because it runs anywhere), mlx (Apple silicon), llama (any GGUF, needs rishi[llama]) and remote (hosted, via fastllm). Pass runtime= only when the model id cannot say for itself.
Try it
Retrieval needs no model; only the answering step does. mk_prompt is the whole contract, so it is worth looking at what the model actually sees.
v = Vault(':memory:')v.note('Late chunking beats naive chunking because context survives the split.')print(mk_prompt('why late chunking?', v.context('late chunking'))[:400])
/Users/71293/code/personal/orgs/vishalakshi/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
[1] Late chunking beats naive chunking because context survives the split.
(source: note:4282b2c0f97d, pages 0–0)
Late chunking beats naive chunking because context survives the split.
---
Question: why late chunking?
/Users/71293/code/personal/orgs/vishalakshi/.venv/lib/python3.13/site-packages/litesearch/core.py:157: UserWarning: 'store': vector search using dtype='f16' but the store is registered as 'f32'. Distances are being computed over reinterpreted bytes — pass dtype= matching the embeddings you inserted.
warnings.warn(f'{tbl.name!r}: vector search using dtype={want!r} but the store is registered '
res = L([AttrDict(node_id='d#1', title='A', breadcrumb='A › B', filename='f', doc_id='d')])test_eq(cited('as [1] shows, and again [1], but not [9]', res).attrgot('node_id'), ['d#1'])
# reasoning arrives three ways: a full pair, an opener cut off at the cap, and — from an MLX chat# template that prefills the opener — a bare closing tag with the reasoning in front of ittest_eq(split_reasoning('<think>weighing [1]</think>\n\nRRF fuses ranks [2].'), ('RRF fuses ranks [2].', 'weighing [1]'))test_eq(split_reasoning('weighing [1]</think>\n\nRRF fuses ranks [2].'), ('RRF fuses ranks [2].', 'weighing [1]'))test_eq(split_reasoning('RRF fuses ranks [2].'), ('RRF fuses ranks [2].', ''))# a section the reasoning weighed and dropped must not come back as a citationtest_eq(cited(split_reasoning('cites [1]</think> cites nothing')[0], res), [])