vishalakshi

one vault for everything you read: web, papers, video, files and code, searchable together and answerable by a local or hosted model

Install

pip install 'vishalakshi[all]'   # + rishi to answer, + kosha & rgapi for code, + mcp for the server
pip install vishalakshi          # vault + acquisition only; no LLM, no MCP

The loop

Everything below runs. The corpus is this repository’s own documentation and source, so the page is reproducible from a clone, and the vault is a throwaway file rather than your real one — Vault() with no argument uses ~/.vishalakshi/vault.db.

from tempfile import mkdtemp
from vishalakshi import Vault

v = Vault(Path(mkdtemp())/'vault.db')
v.enc.note
/Users/71293/code/personal/orgs/vishalakshi/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
'minishlab/potion-multilingual-128M (256d, float16)'

Ingestion is one call per kind of thing. Start with what is already on disk — no network, no model:

v.add_dir('..')                 # README.md and every notebook under nbs/
v.note('federate fuses the legs by rank because they share no vector space: the vault embeds '
       'prose, kosha embeds identifiers, ripgrep embeds nothing.', tags=['retrieval', 'design'])
v.connect()                      # the entity graph over everything now in the vault
{'entities': 1191,
 'mentions': 2548,
 'edges': 1062,
 'windows': 820,
 'resolved': {'merged': 629,
  'by_ann': 214,
  'by_lexical': 415,
  'edges': 385,
  'entities': 1191,
  'resolvable': 1191,
  'canonical': 562}}

connect() is what the associative leg of retrieval runs on, so it goes after a batch of ingests rather than after each one — it reads the whole store.

The rest of acquisition needs the network, which is the only reason these are not run here:

v.web('late chunking retrieval', n=5)           # search the web, read the hits, file them
v.arxiv('2409.04701')                           # a paper, full text
v.youtube('https://youtu.be/aircAruvnKk')       # a talk, transcribed
v.code('~/src/litesearch')                      # a source tree, as documents
v.crawl('https://tdd.cat')              # or a file, a directory, an arXiv id, a YouTube URL
[2026-08-06 21:07:52] INFO: Fetched (200) <GET https://arxiv.org/abs/2409.04701> (referer: https://www.google.com/)
[2026-08-06 21:07:52] INFO: Fetched (200) <GET https://weaviate.io/blog/late-chunking> (referer: https://www.google.com/)
[2026-08-06 21:07:52] INFO: Fetched (200) <GET https://arxiv.org/pdf/2409.04701> (referer: https://www.google.com/)
[2026-08-06 21:07:52] INFO: Fetched (200) <GET https://jina.ai/news/late-chunking-in-long-context-embedding-models/> (referer: https://www.google.com/)
[2026-08-06 21:07:53] INFO: Fetched (200) <GET https://github.com/jina-ai/late-chunking> (referer: https://www.google.com/)
[2026-08-06 21:07:53] INFO: Fetched (200) <GET https://tdd.cat/> (referer: https://www.google.com/)
[2026-08-06 21:07:53] INFO: Fetched (200) <GET https://tdd.cat/archive/> (referer: https://www.google.com/)
[2026-08-06 21:07:54] INFO: Fetched (200) <GET https://tdd.cat/> (referer: https://www.google.com/)
[2026-08-06 21:07:54] INFO: Fetched (200) <GET https://tdd.cat/stats/> (referer: https://www.google.com/)
[2026-08-06 21:07:54] INFO: Fetched (200) <GET https://tdd.cat/rss.xml> (referer: https://www.google.com/)
[2026-08-06 21:07:54] INFO: Fetched (200) <GET https://tdd.cat/2026-08-04/> (referer: https://www.google.com/)
[2026-08-06 21:07:54] INFO: Fetched (200) <GET https://tdd.cat/2026-08-03/> (referer: https://www.google.com/)
[2026-08-06 21:07:55] INFO: Fetched (200) <GET https://tdd.cat/2026-08-02/> (referer: https://www.google.com/)
[2026-08-06 21:07:55] INFO: Fetched (200) <GET https://tdd.cat/2026-08-01/> (referer: https://www.google.com/)
[2026-08-06 21:07:55] INFO: Fetched (200) <GET https://tdd.cat/2026-07-30/> (referer: https://www.google.com/)
[{'doc_id': 'ca0734f08c3a2af4', 'title': 'https://tdd.cat/', 'kind': 'web', 'nodes': 2, 'chunks': 2}, {'doc_id': 'a1d93f33dff80a49', 'title': 'https://tdd.cat/archive/', 'kind': 'web', 'nodes': 2, 'chunks': 1}, {'doc_id': 'ca0734f08c3a2af4', 'title': 'https://tdd.cat/', 'skipped': 'already ingested; pass force=True'}, {'doc_id': '404fecf0dc5f0f25', 'title': 'https://tdd.cat/stats/', 'kind': 'web', 'nodes': 2, 'chunks': 1}, {'doc_id': '1cabfb266b4428de', 'title': 'https://tdd.cat/rss.xml', 'kind': 'web', 'nodes': 2, 'chunks': 32}, {'doc_id': '44006e2ad0153874', 'title': 'https://tdd.cat/2026-08-04/', 'kind': 'web', 'nodes': 2, 'chunks': 2}, {'doc_id': '10f419e5507fd994', 'title': 'https://tdd.cat/2026-08-03/', 'kind': 'web', 'nodes': 2, 'chunks': 3}, {'doc_id': '632bac90de98875d', 'title': 'https://tdd.cat/2026-08-02/', 'kind': 'web', 'nodes': 2, 'chunks': 3}, {'doc_id': '77f5ff98daeb2ffd', 'title': 'https://tdd.cat/2026-08-01/', 'kind': 'web', 'nodes': 2, 'chunks': 3}, {'doc_id': '6eef8f3cf6502eb5', 'title': 'https://tdd.cat/2026-07-30/', 'kind': 'web', 'nodes': 2, 'chunks': 3}]

v.grab(target) does whichever of those the target names, which is the one call a script or an agent needs.

Then ask it something. ask retrieves first and answers from what came back, so the model never sees anything but your own corpus:

r = v.ask('why are rankings fused instead of distances?', model='mlx-community/Qwen3-4B-4bit')
print(r.model, '·', r.runtime, '·', len(r.thinking), 'chars of thinking split off')
print(r.answer)
mlx-community/Qwen3-4B-4bit · mlx · 0 chars of thinking split off
The reason rankings are fused instead of distances is that the different legs of the retrieval system (prose, identifiers, and file searches) use distinct embedding strategies and share no common vector space. This means their distances cannot be directly combined, but their rankings can. The system uses **reciprocal rank fusion (RRF)** to combine rankings from these legs, as ranking is the only metric that survives changes in encoding methods. 

This is explicitly stated in the vault's design: "federate fuses the legs by rank because they share no vector space" [3], and elaborated in section 6, which explains that "the legs share no vector space, so federate fuses their *rankings* with RRF, never their distances" [6]. 

The three legs (prose, kosha, ripgrep) use different encoders and thus cannot combine distances directly. Instead, rankings are fused using RRF, ensuring compatibility across different embedding strategies. Ranking is the only metric that survives changes in encoding methods, and this approach leverages the fact that ranking is the same mechanism used to combine full-text search (FTS) with vector-based searches.

Every [n] in the answer resolves to a node_id you can read. That round trip is the point: an answer you can check, not one you have to believe.

for c in r.cited: print(c['n'], c['breadcrumb'])
print()
print(v.read(r.cited[0]['node_id'])['text'][:400])   # the exact text behind the claim
3 index › What retrieval actually returns
6 index › Code: kosha, ripgrep, and federated search

`v.context(q)` is the primitive underneath `ask`. It returns whole **sections** — the unit worth
reading — not fragments:

c = v.context('why are rankings fused instead of distances?', sections=4, related=4)
for s in c.results: print(f'{len(s.text):5}  {s.breadcrumb}')

for s in c.related: print(f'{s.via:6}  {s.breadcrumb}')

`results` are the sections that answer the question. `related` are secti

What retrieval actually returns

v.context(q) is the primitive underneath ask. It returns whole sections — the unit worth reading — not fragments:

c = v.context('why are rankings fused instead of distances?', sections=4, related=4)
for s in c.results: print(f'{len(s.text):5}  {s.breadcrumb}')
 3968  README › The loop
 1300  README › What retrieval actually returns
  722  index › What retrieval actually returns
 3174  README › Code: kosha, ripgrep, and federated search
for s in c.related: print(f'{s.via:6}  {s.breadcrumb}')
graph   index › {'kind': 'data', 'records': 120, 'endpoint': '.../api/bff/products'} › Encoders
vector  index › The loop

results are the sections that answer the question. related are sections reached by association — along the entity graph (via='graph') or by embedding similarity (via='vector') — which is how you find the thing you did not know to search for.

c.encoder
'minishlab/potion-multilingual-128M (256d, float16)'

c.encoder always says which embedder answered, because degrading from real semantics to hashing changes what the results mean.

Everything in one corpus

Every document carries a kind — web, pdf, arxiv, youtube, file, code, data, note, or whatever litesearch’s parser called the file. Filter when you want to, don’t when you don’t:

v.stats()
{'docs': 27,
 'nodes': 113,
 'chunks': 734,
 'entities': 1191,
 'by_kind': {'web': 14,
  'notebook': 8,
  'youtube': 1,
  'txt': 1,
  'note': 1,
  'md': 1,
  'arxiv': 1},
 'encoder': 'model2vec',
 'path': '/var/folders/kg/9vdw4mdd1fs58svgh4k1qhr09x7dqh/T/tmp15e4qu8e/vault.db'}
len(v.find('rank fusion')), len(v.find('rank fusion', kind='note')), len(v.find('rank fusion', kind='md,notebook'))
(10, 1, 10)

The filter is a SQL WHERE pushed into litesearch’s search, not a pass over the results afterwards, so a narrow filter over a large vault still returns a full page of hits.

Notes are ordinary documents, deliberately. The graph, the clusters and context() all see them for free, so what you concluded about a corpus comes back next to the evidence you concluded it from.

L(v.find('rank fusion', kind='note')).attrgot('content')
['federate fuses the legs by rank because they share no vector space: the vault embeds prose, kosha embeds identifiers, ripgrep embeds nothing.']

Knowing what you have

L(v.sources()).map(lambda d: (d['kind'], d['title'], d['source']))[:6]
[('web', 'https://tdd.cat/', 'https://tdd.cat/'), ('web', 'https://tdd.cat/archive/', 'https://tdd.cat/archive/'), ('web', 'https://tdd.cat/stats/', 'https://tdd.cat/stats/'), ('web', 'https://tdd.cat/rss.xml', 'https://tdd.cat/rss.xml'), ('web', 'https://tdd.cat/2026-08-04/', 'https://tdd.cat/2026-08-04/'), ('web', 'https://tdd.cat/2026-08-03/', 'https://tdd.cat/2026-08-03/')]
L(v.toc()).attrgot('title')
['README', 'robots', 'index', '00 core', '01 acquire', '02 ask', '03 code', '04 cli', '05 mcp', 'index', 'federate fuses the legs by rank because they share no vector space: the vault em', 'arXiv:2409.04701v3 [cs.CL] 7 Jul 2025 LATE CHUNKING: CONTEXTUAL CHUNK EMBED-', '[2409.04701] Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models', 'Late Chunking: Balancing Precision and Cost in Long Context Retrieval | Weaviate', 'Late Chunking in Long-Context Embedding Models', 'GitHub - jina-ai/late-chunking: Code for explaining and evaluating late chunking (chunked pooling) · GitHub', 'Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models', 'But what is a neural network? | Deep learning chapter 1', 'https://tdd.cat/', 'https://tdd.cat/archive/', 'https://tdd.cat/stats/', 'https://tdd.cat/rss.xml', 'https://tdd.cat/2026-08-04/', 'https://tdd.cat/2026-08-03/', 'https://tdd.cat/2026-08-02/', 'https://tdd.cat/2026-08-01/', 'https://tdd.cat/2026-07-30/']
L(v.map().clusters).map(lambda c: (c['size'], c['label']))
[(8, 'gmtthe, tdd, diff, daily'), (8, 'g1n'), (8, 'cx5, ito, gdy, q8g'), (8, 'tf0, lxx, q5v, jx5'), (8, 'ivv, eiqsr, ihh, eiq3'), (8, 'dceu, nuh, i54, ggkba'), (8, 'j5u, er0, dps, g_aaa'), (8, 'ii3, g0d, jlwc, tdgg'), (8, 'neuron, image, neurons, layer'), (7, 'n5g, zq2, jh1, tty'), (6, 'dle, pui, dx7, tqr'), (6, 'onb, ym4, zm79p, ohj'), (6, 'dee, qhr, qqr, xer'), (6, 'edges, hope, neural, network'), (6, 'curation, signal, papers, strong'), (5, 'vky, ruu, re2st, lll'), (5, 'lxbt, sox, kbbb, fyov'), (5, 'wh6, gee, l0g, row'), (5, 's1q, kpv, gf5kt, zsu'), (5, 'm0o, wszz, dgg, dzz'), (5, 'layer, way, activations, corresponds'), (5, 'applied, resulting, contextual, information'), (5, 'browse, retailer, www, dairy'), (5, 'smaller, representation, applications, embedding'), (4, 'rir, cwt, cl7, l2x'), (4, 'wh7, ckz, onu, gpl'), (4, 'cj1, kwp, w06n, ngv'), (4, 'dxx, edd, k9r, khp'), (4, 'f7o, gus, xlh, vz1'), (4, 'pxj, yzm, xvi, pz7'), (4, 'bz9g, z9g, cz9g, sfo'), (3, 'ogd, mwds, ruumjj'), (3, 'arvu, kl9p, hojqi, eu4'), (3, 'jnn, zil, shf, yi1'), (3, 'jhb, ehc, hwr, hee'), (3, 'iin0, v3sf_, stc, lmr'), (3, 'xqv, ior, dbu, ctu'), (3, 'ejn, m3p, lb6, oty'), (3, 'ij1, bgu, c5r, hsu'), (3, 'wh_, bkkx'), (3, 'hwx, gq3, ak4, c3o'), (3, 'hxhn, eja, kbc, _h6ah'), (3, 'cb_, nhh, h2t, hcd'), (3, 'dhh, hqr, sss, fgg_'), (3, 'bqj4, j_b, sui'), (3, 'gmtthe, jul, tdd, diff'), (3, 'biases, setting, weights, hand'), (3, 'subscribe, stay, videos, video'), (3, 'harness, kimi, curation, signal'), (3, 'adds, build, step, tags'), (3, 'rgapi, server, acquisition, mcp'), (3, 'filling, accumulates, restarts, plus'), (3, 'bare, rejected, four, marketing'), (3, 'gets, price, buried, record'), (3, 'hash_embed, failing, air, deterministic'), (3, 'answered, always, degrading, association'), (3, 'throwaway, clone, documentation, tempfile'), (3, 'changelog, params, remind, action'), (3, 'design, entity, notebook, tags'), (3, 'badly, including, indexed, edit'), (3, 'client, speaks, mcpservers, codex'), (3, 'optimisation, thread, survive, width'), (3, 'luna, fastllm, cpp, gemini'), (3, 'worth, fragments, primitive, underneath'), (3, 'aircaruvnkk, src, transcribed, youtu'), (3, 'fts, survives, contributed, reports'), (3, 'cmd, docments, wrapper, date'), (3, 'afterwards, sql, narrow, notebook'), (3, 'round, claim, trip, cited'), (3, 'tier, nid, later, months'), (3, 'prepare, nbdev, modules, notebooks'), (3, 'fills, installed, packages, indexes'), (3, 'citations, block, reasoning, discards'), (3, 'eggs, range, limit, free'), (3, 'retrieves, came, answers, qwen3'), (3, 'part, jina, lc2, exploration'), (3, 'prose, embeds, identifiers, nothing'), (3, 'dims, ndim, store, return'), (2, 'yvo, rqg'), (2, 'gmtthe, jul, tdd, daily'), (2, 'mdg, gwh, mux, jx4s'), (2, 'yyw, qtz, npn, q2z3'), (2, 'mwg, vkx, omme, ddd'), (2, 'o7ov, klz'), (2, 'fcy, wik, s5h, sp5'), (2, 'eee, ccc, _vc, vurh'), (2, 'vmttt, g8u, ojj, fdgg'), (2, 'bu8, i3y, nfg, njmm'), (2, 'l9g, jr6, n5n, hub'), (2, 'kll, rll'), (2, 'ai9awshuh, ptf, iae, agvv'), (2, 'ors, usjc, gve, bzto'), (2, 'mz0ru, ico, afc'), (2, 'ehac, gmosh, vur, _ru'), (2, 'ins, rbo, m0a, g_oc'), (2, 'x2m, qzdx, yul, fjd'), (2, 'xiy, tl1dv, jnt'), (2, 'mke, zcl, wvrdb, yz7'), (2, 'tuesday, august, tdd, diff'), (2, 'ucmo, x9d'), (2, 'vgw'), (2, 'nhmd, dic, dccfe, w5s'), (2, 'oz3, cnd, s7p, d4b'), (2, 'cvu, ua21, jsy, jul0g'), (2, '_sl, v3aj, cur, nknd'), (2, 'group-62'), (2, 'qwi, kcv, gee'), (2, 'mdzke4, c3ob, qqq, jkk'), (2, 'length2, length3, length, stream'), (2, 'hidden, number, going, neurons'), (2, 'deep, neural, networks, music'), (2, 'either, analogy, biological, train'), (2, 'matrix, sigmoid, biases, function'), (2, 'likely, compressed, portions, shorter'), (2, 'docs, doc_id, enc, none'), (2, 'gpu, sunday, llm, curation'), (2, 'degree, rrf_all, gitignore, callers'), (2, 'vishalakshi_offline, forces, hashing, encoder'), (2, 'writes, stops, involved, dead'), (2, 'concluded, deliberately, clusters, ordinary'), (2, 'webassembly, bridging, networking, game'), (2, 'reimplementing, javascript, dependencies, pain'), (2, 'app, icon, access, offline'), (2, 'storage, interaction, required, naive'), (2, 'group, must, user, shines'), (2, 'consist, similar, tested, beir'), (2, 'sep, jul, revised, submitted'), (2, 'get_pdf, path_or_url, force, verify'), (2, 'negative, positive, weighted, sum'), (2, 'stacks, requirements, helps, optionality'), (2, 'force, fetched_at, meta, return'), (2, 'float, seconds, params, action'), (2, 'google, fetched_at, chars, verify'), (2, 'harvest, export, apis, add_file'), (2, 'contextual, information, long, approach'), (2, 'ttl, urlparse, items, endpoint')]

sources() is where provenance lives: it is kept at ingest, so months later it still says why a document is in your corpus — the query that found it, when, which tier fetched it. related(node_id) answers the other question, “what else in here reads like this”:

nid = v.find('rank fusion')[0]['node_id']
L(v.related(nid, limit=4)).attrgot('breadcrumb')
['index › Code: kosha, ripgrep, and federated search', 'index › Code: kosha, ripgrep, and federated search', 'README › What retrieval actually returns', '03 code']

CLI

vishalakshi grab https://example.com/post     # or a file, a directory, an arXiv id, a YouTube URL
vishalakshi web "late chunking retrieval"     # search and ingest in one step
vishalakshi note "..." --tags retrieval
vishalakshi connect                           # build the graph after a batch of adds
vishalakshi context "why does late chunking help"
vishalakshi ask "why does late chunking help"
vishalakshi sources; vishalakshi map; vishalakshi toc

Every Vault method is a command, and every command’s --help is generated from the method’s own signature and docments — so vishalakshi <cmd> --help is never out of date, and there is no wrapper per command in the source:

from vishalakshi.cli import CMDS, cmd
from fastcore.script import anno_parser

len(CMDS), CMDS[:8]
(38,
 ['stats', 'find', 'sections', 'context', 'ask', 'explain', 'read', 'related'])
anno_parser(cmd('find'), prog='vishalakshi find').print_help()
usage: vishalakshi find [-h] [--limit LIMIT] [--kind KIND] q

Chunk-level hybrid search (FTS5 + vectors, RRF-fused), each hit carrying its breadcrumb.

positional arguments:
  q              query

options:
  -h, --help     show this help message and exit
  --limit LIMIT  hits to return (default: 10)
  --kind KIND    restrict to one or more KINDS ('note' or 'note,web')

$VISHALAKSHI_VAULT picks the vault file, $VISHALAKSHI_MODEL the model ask uses, $VISHALAKSHI_OFFLINE forces the hashing encoder.

MCP

vishalakshi-mcp exposes the vault to any MCP client — Claude Code, Codex, or anything else that speaks the protocol:

{"mcpServers": {"vishalakshi": {"command": "vishalakshi-mcp",
                                "env": {"VISHALAKSHI_VAULT": "~/.vishalakshi/vault.db"}}}}
from vishalakshi.mcp import TOOLS
len(TOOLS), TOOLS
(32,
 ['stats',
  'find',
  'sections',
  'context',
  'ask',
  'read',
  'related',
  'toc',
  'map',
  'sources',
  'grab',
  'url',
  'web',
  'arxiv',
  'youtube',
  'add_file',
  'add_dir',
  'note',
  'connect',
  'forget',
  'apis',
  'harvest',
  'watch',
  'watches',
  'poll',
  'unwatch',
  'index_code',
  'code_search',
  'symbol',
  'where_to_add',
  'grep',
  'federate'])

Those tools are built from the same Vault methods the CLI uses, so each tool’s schema and description are the method’s signature and docstring. context and find for reading, ask for answering, grab / web / arxiv / youtube / add_file / add_dir / note for filling it, plus toc, map, sources, related, read, connect and forget. An agent that can write notes back into the same vault it reads from accumulates rather than restarts.

Models and backends

ask goes through rishi, which has four backends. rishi picks one from the shape of the model id, so a bare marketing name (gemma-3-4b-it-int4) matches nothing and is rejected rather than guessed at. Name a real id, use an alias, or say the runtime.

runtime what it is model ids look like needs
litert Google LiteRT, CPU — the default litert-community/…, *.litertlm nothing; runs anywhere
mlx Apple silicon mlx-community/… macOS on ARM
llama llama.cpp, any GGUF *-GGUF, *.gguf pip install 'rishi[llama]'
remote hosted, via fastllm gpt-5.6-luna, gemini-… an API key 
from vishalakshi.ask import MODELS, dflt_model, resolve_model
dflt_model
'litert-community/gemma-4-E2B-it-litert-lm'
L(MODELS.items()).map(lambda kv: (kv[0], *kv[1][1:]))
[('gemma-e2b', 'litert', 'default: ~2GB, CPU, runs anywhere'), ('gemma-e4b', 'litert', 'larger LiteRT build'), ('gemma-12b', 'litert', 'biggest LiteRT build'), ('qwen-4b-mlx', 'mlx', 'Apple silicon'), ('ornith-9b-mlx', 'mlx', 'Apple silicon'), ('qwen-4b-gguf', 'llama', 'llama.cpp, needs rishi[llama]'), ('gemma-4b-gguf', 'llama', 'llama.cpp, needs rishi[llama]'), ('gpt-5.6-luna', 'remote', 'hosted, needs OPENAI_API_KEY'), ('gpt-5.6-sol', 'remote', 'hosted, needs OPENAI_API_KEY')]
(resolve_model('qwen-4b-mlx'),                         # alias -> id + runtime
 resolve_model('mlx-community/Qwen3-4B-4bit'),         # full id, runtime inferred
 resolve_model('my-local.gguf', runtime='llama'))      # say it outright when the id can't
(('mlx-community/Qwen3.5-4B-MLX-4bit', 'mlx'),
 ('mlx-community/Qwen3-4B-4bit', 'mlx'),
 ('my-local.gguf', 'llama'))
from fastcore.test import test_fail
test_fail(lambda: resolve_model('gemma-3-4b-it-int4'), contains='backend')

A reasoning model’s <think> block is split off into r.thinking rather than left in r.answer — it names sections it then discards, and citations are read off the answer.

The vault never needs the network to retrieve. Only answering with a hosted model does.

Harvest: read a page’s API, not its HTML

Listing, product and dashboard pages render from an internal JSON API. fossick can watch a page and capture those calls; the vault turns them into searchable records.

v.apis('https://www.example-retailer.com/browse/dairy')
# [0] .../api/bff/products?page=1   records: 24   {"results":[{"sku":...

v.harvest('https://www.example-retailer.com/browse/dairy', pages=5)
# {'kind': 'data', 'records': 120, 'endpoint': '.../api/bff/products'}

v.find('free range eggs')     # each product is its own retrievable section

Each record becomes a ## section, so it gets its own tree node and breadcrumb rather than being buried in one blob. session=True captures through your logged-in Chrome, so pages behind a login work too. add_records(...) files a list of dicts you already have:

v.add_records([dict(sku='A1', name='Free range eggs', price=4.5),
               dict(sku='B2', name='Whole milk', price=2.1)], 'dairy demo')
{'doc_id': '0b8b93f9d53bd84e',
 'title': 'dairy demo',
 'kind': 'data',
 'nodes': 3,
 'chunks': 3}
L(v.find('free range eggs', limit=2)).attrgot('breadcrumb')
["index › {'kind': 'data', 'records': 120, 'endpoint': '.../api/bff/products'}", 'dairy demo › Free range eggs']

Watches: keeping it current

v.watch('https://example.com/changelog', action='url',     every='6h')
v.watch('late chunking retrieval',       action='web',     every='1d', n=5)
v.watch('Re-read the eval numbers',      action='remind',  every='1w')

L(v.watches()).map(lambda w: (w['action'], w['target'][:34], w['every'], w['params']))
[('url', 'https://example.com/changelog', 21600.0, {}), ('web', 'late chunking retrieval', 86400.0, {'n': 5}), ('remind', 'Re-read the eval numbers', 604800.0, {})]
v.poll()      # run everything due; failures are recorded on the row, never raised

poll() is the tick — call it from cron, a scheduler, or a frontend button. An action is just the name of an acquisition method, so anything you can file once you can file on a schedule; remind writes a note with no network involved. One dead URL never stops the loop.

Encoders

The vault wants a real embedder and will fetch a small model2vec one by default. Where that is impossible — an air-gapped box, a blocked registry — it degrades to litesearch’s deterministic hash_embed rather than failing, and says so in stats()['encoder'] and on every context() result. Retrieval still works; it is lexical rather than semantic, and you should know which you are getting.

Vault(':memory:', offline=True).enc.note      # never attempt a download — also the CI default
'char-n-gram hashing (256d) — lexical only; pass encoder= or restore network access for real semantics'
Vault(encoder='minishlab/potion-science-32M')   # pick a different one

Both encoders are stored at float16, which is litesearch’s own default width. That is not a size optimisation: Database.context does not thread a dtype= down to its section search, so a float32 store is read back there as float16 — the vectors survive, the distances do not, and retrieval silently degrades to keyword ranking.

Development

The notebooks in nbs/ are the source; the modules are generated.

pip install -e '.[all]'
nbdev-prepare