skill

one vault for everything you have read: web pages, papers, video, files, code and your own notes in one SQLite

Getting started

from pathlib import Path
from vishalakshi.skill import vault
v = vault()                      # $VISHALAKSHI_VAULT, else ~/.vishalakshi/vault.db
v.add(Path('~/notes').expanduser())   # a directory, a file, or text; `~` is not expanded for you
v.grab('1706.03762')             # an arXiv id, YouTube link, GitHub repo, PDF link or path
v.note('what I concluded', tags=['retrieval'])
v.note('what I concluded', title='late chunking', key='late-chunking')   # same key: one document, replaced

Adding a source again re-reads it only when its content changed, and the result’s changed says whether it did.

Reading it back

v.context(q)                     # whole sections plus what they connect to; hand this to a model
v.ask(q)                         # answer with [n] citations -> node_ids
v.search(q, limit=10)            # chunk hits
v.sections(q, limit=5)           # ranked sections
v.read(node_id)                  # section behind a citation

kind= filters by how a document arrived: v.search(q, kind='pdf'), or several as 'note,web'. The kinds are web, pdf, arxiv, youtube, file, code, data, note and image. Every row carries age (seconds since it was filed) and stale, and stale rows sort last. v.mark_stale(ref, reason=) sets the mark, and adding the document again with new content clears it.

Named documents

v.ask_doc(['a.py', 'b.py'], 'what does a do that b does not?')
v.ask_doc(path, 'what is owed and by when?', schema='amount:float, due:str')

A path on disk is read even when it is not in the vault yet. The named documents share one character budget, doc_chars.

Paperwork

v.categorize(ref, llm='never')   # cue table first; a model only when the cues cannot decide
v.extract(ref)                   # doctype picks the schema
v.extract(ref, schema='vendor:str, total:float, due_date:str')
v.extract_all(doctype='invoice')

decisive=True means the cues decided without a model.

Pictures

v.add_image(path, labels=['Superb Fairywren'])   # metadata is the text; pixels stay on disk
v.add_images(dir)
v.label_images(dir, 'aussie-birds')              # anya classifies, the vault files the result
v.tag_image(ref, labels=[...], caption='...')    # write a later model run onto a picture
v.images(); v.search(q, kind='image')

Pictures have kind='image'. The searchable text is the filename, folder, dimensions, camera, capture date and GPS position, with any labels or caption. label_images needs anya.

Code

v.index_code(dir)                # kosha: AST chunks, symbols, call graph
v.code_search(q); v.symbol(name); v.where_to_add(desc); v.grep(pat, dir)
v.federate(q, dir=dir)           # vault + kosha + ripgrep, fused by rank

Once .kosha/code.db exists, context and ask add code sections on their own.

Knobs

Retrieval defaults are litesearch’s. Reach for:

rerank=True on search / sections / context when precision matters more than latency (needs vishalakshi[rerank])
v.shelf(name) to keep two collections apart, so a large one does not crowd a small one out of the rankings
llm= (auto, always or never) for when categorize and extract may ask a model
ask(q, pii=): default local when local_ok() (the local runtime is importable), else redact; never off. The result’s policy says which ran; a host sets vishalakshi.ask.LOCAL_OK = False to force redact

Marks and feedback

v.mark_noisy(doc_id, reason='site furniture')
v.mark_pii(doc_id, reason='address book'); v.mark_not_pii(doc_id, reason='my own invoice')
v.pii(doc_id, ner=True)          # read `scanned_ner`, not the count alone
v.level(doc_id)                  # Open, Personal or Local only; an agent may raise one, never lower it
v.send_check(refs, model=m)      # what a model would get: whole, masked or withheld, and `why`
v.suggest_noisy(k=20); v.accept_noisy(k=10)
v.learn()                        # log every ask as feedback

Marks live in doc_marks and survive the document being added again. suggest_noisy is 0.988 AUC. Leave fit_ranker / use_ranker off unless your corpus says otherwise (evals/RESULTS.md).

connect(), which needs a model, builds the typed entity graph (LLM refers_to/defines edges) and the topic nodes; graph_search walks it, map / topic_tree read topics. Run connect() after a batch of ingests.

Watches

v.watch(url, every='6h')                                   # kind='url': re-read when it changed
v.watch(dir, kind='folder', every='1h', instructions='flag any print()', pattern='*.py')
v.watch('re-read the evals', kind='remind', every='1w')    # one note per reminder
v.watches(); v.poll()                                      # poll drains due work and housekeeps stale documents

kind is action by another name; a folder watch fires with added, changed, removed and its instructions, verbatim.

Notes

stats / sources / toc inspect the vault. Ingest is content-addressed. Sanskrit turns on vidyut lemmas and Monier-Williams glosses once.

forget, drop_shelf, unwatch and pause are not in allow; call them yourself when you mean to.

This page is a pyskill. A pyskill is a cheat sheet that an agent loads with read_skill.

The markdown cell above is the module docstring. list_pyskills() reads it without importing the module. It takes the first paragraph as the summary.

allow registers the vault methods that a sandboxed agent may call. READ lists the methods that read the vault. WRITE lists the methods that add to it.

READ and WRITE leave out the methods that delete (forget, drop_shelf) and the methods that stop watches (unwatch, pause). An agent cannot call them.