# llm


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

This module connects the tutor to a model. The tutor needs only one
function, `ask(prompt, sp)`, which returns text. The module also parses
JSON replies and gives
[`Fake`](https://vedicreader.github.io/abhyasa/llm.html#fake), a model
for tests that runs offline.

Import the tools that this module uses.

## The model

Each surface gives its own `ask`.

<table>
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr>
<th>surface</th>
<th><code>ask</code></th>
</tr>
</thead>
<tbody>
<tr>
<td>web app, notebook</td>
<td><a
href="https://vedicreader.github.io/abhyasa/llm.html#rishi_ask"><code>rishi_ask</code></a>:
a new <code>Chat</code> of the model layer for each call, on
<code>$ABHYASA_MODEL</code> (default <code>claude/sonnet</code>)</td>
</tr>
<tr>
<td>the ramabana terminal agent</td>
<td><code>agent.oneshot</code>, so the routing of the agent applies</td>
</tr>
<tr>
<td>tests</td>
<td><a
href="https://vedicreader.github.io/abhyasa/llm.html#fake"><code>Fake</code></a></td>
</tr>
</tbody>
</table>

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L18"
target="_blank" style="float:right; font-size:smaller">source</a>

### rishi_ask

``` python
def rishi_ask(
    model:str=None
):
```

*An `ask(prompt, sp='')` on a `rishi.Chat` (the model layer) for
`model`, else `$ABHYASA_MODEL`, else `DFLT_MODEL`.*

## JSON replies

[`parse_json`](https://vedicreader.github.io/abhyasa/llm.html#parse_json)
finds the JSON in a reply.
[`ask_json`](https://vedicreader.github.io/abhyasa/llm.html#ask_json)
asks again one time when the reply is not valid JSON.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L44"
target="_blank" style="float:right; font-size:smaller">source</a>

### ask_json

``` python
def ask_json(
    ask, prompt:str, sp:str='', tries:int=2
):
```

*Ask for JSON and parse it. On a bad reply, ask one more time with the
parse error.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L30"
target="_blank" style="float:right; font-size:smaller">source</a>

### parse_json

``` python
def parse_json(
    s:str
):
```

*The first JSON object or array in `s`, also inside a
\`\``fence. Raise`ValueError\` when there is none.*

[`parse_json`](https://vedicreader.github.io/abhyasa/llm.html#parse_json)
reads JSON in a fence, JSON inside other text, and arrays. A fence
inside a JSON string does not break it. It fails when there is no JSON.

``` python
test_eq(parse_json('```json\n{"a": 1}\n```'), {'a': 1})
test_eq(parse_json('Here it is: {"a": [1, 2]} thanks'), {'a': [1, 2]})
test_eq(parse_json('[{"x": 1}]'), [{'x': 1}])
test_fail(lambda: parse_json('nothing'), contains='no JSON')
test_eq(parse_json('{"answer": "Use ```python\\nx = 1\\n``` here."}'), {'answer': 'Use ```python\nx = 1\n``` here.'})
test_eq(parse_json('Sure. {"a": 1} and then {"b": 2}'), {'a': 1})
```

## A model for tests

[`Fake`](https://vedicreader.github.io/abhyasa/llm.html#fake) replies
from a dict of rules. The first rule with a key in the prompt gives the
reply. It sends a dict or a list reply as JSON. It keeps each call in
`calls`.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L55"
target="_blank" style="float:right; font-size:smaller">source</a>

### Fake

``` python
def Fake(
    rules:dict=None, dflt:str='{}'
):
```

*A stand-in model for tests:
`Fake({'grade': {...}, 'question': {...}})`.*

A matching rule gives its reply, and
[`Fake`](https://vedicreader.github.io/abhyasa/llm.html#fake) records
the call.

``` python
f = Fake({'GRADE': dict(correct=True)})
test_eq(ask_json(f, 'GRADE this'), dict(correct=True))
test_eq(len(f.calls), 1)
```

After a reply that is not JSON,
[`ask_json`](https://vedicreader.github.io/abhyasa/llm.html#ask_json)
asks again and gets the JSON.

``` python
bad = Fake(dict(X=lambda p, sp: 'oops' if 'not valid' not in p else '{"ok": 1}'))
test_eq(ask_json(bad, 'X'), {'ok': 1})
```

## A model for each task

The tutor does five tasks, and each task can use its own model. A quick
model can grade and give hints. A strong model can write questions and
lessons. The Settings page saves the choice.
[`Router`](https://vedicreader.github.io/abhyasa/llm.html#router) reads
it on each call, so a change has an effect on the next call.

<table>
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr>
<th>task</th>
<th>what the model does</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>question</code></td>
<td>writes a question and its hidden model answer</td>
</tr>
<tr>
<td><code>grade</code></td>
<td>reads the conversation and grades it</td>
</tr>
<tr>
<td><a
href="https://vedicreader.github.io/abhyasa/rama.html#hint"><code>hint</code></a></td>
<td>gives the next hint</td>
</tr>
<tr>
<td><code>lesson</code></td>
<td>teaches the smallest step after a miss</td>
</tr>
<tr>
<td><a
href="https://vedicreader.github.io/abhyasa/sources.html#extract"><code>extract</code></a></td>
<td>finds the concepts in a text</td>
</tr>
</tbody>
</table>

[`model_ask`](https://vedicreader.github.io/abhyasa/llm.html#model_ask)
gives an `ask` for any model name. Install the `models` extra
(`pip install abhyasa[models]`) to use the full list of models. With the
extra, the name can be any model in that list: Claude Code, Codex,
OpenAI, Copilot, Gemini, or a model on this machine (LiteRT, mlx,
llama.cpp, Ollama). The extra finds the model for the name and gives a
backend of the model layer. Without the extra, the name goes to the
model layer as it is. Each model has one lock, so the calls take turns.
A backend from the full list refuses a second call while one call runs.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L81"
target="_blank" style="float:right; font-size:smaller">source</a>

### model_ask

``` python
def model_ask(
    name:str
):
```

*An `ask(prompt, sp='')` for model `name`, made once and kept.*

## Names for the model list

The Settings page shows a list of models. Each item has a name that
people know, such as “Claude Sonnet” or “GPT-5.5”. The model id, such as
`claude/sonnet`, is the value that the settings keep.
[`model_label`](https://vedicreader.github.io/abhyasa/llm.html#model_label)
makes the name from the id. `PROVIDERS` gives the name of each group in
the list.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L109"
target="_blank" style="float:right; font-size:smaller">source</a>

### model_label

``` python
def model_label(
    value:str, provider:str='', repo:str=''
)->str:
```

*A name that people know for the model id `value`; `repo` is the
download of a local model, if known.*

Each kind of id gets a plain name. A Claude Code tier gets the word
Claude. A local model says so.

``` python
test_eq(model_label('claude/sonnet'), 'Claude Sonnet')
test_eq(model_label('claude/claude-opus-5-5'), 'Claude Opus 5.5')
test_eq(model_label('anthropic/claude-haiku-4-5-20251001'), 'Claude Haiku 4.5')
test_eq(model_label('codex/gpt-6-astra'), 'GPT-6 Astra')
test_eq(model_label('openai/gpt-5.4-mini'), 'GPT-5.4 Mini')
test_eq(model_label('gemini/gemini-3.5-flash'), 'Gemini 3.5 Flash')
test_eq(model_label('qwen-4b', 'mlx', 'mlx-community/Qwen3.5-4B-MLX-4bit'), 'a local model: Qwen 3.5 4B')
test_eq(model_label('gemma-e4b', 'litert', 'litert-community/gemma-4-E4B-it-litert-lm'), 'a local model: Gemma 4 E4B')
test_eq(model_label('ollama/qwen3:8b'), 'a local model: Qwen 3 8B')
test_eq(model_label('my-model', 'azure'), 'my-model')
```

[`catalog`](https://vedicreader.github.io/abhyasa/llm.html#catalog)
lists the models that can run here, for the Settings page. With the
`models` extra, the list is the full list of models. Each row gets a
`name` from
[`model_label`](https://vedicreader.github.io/abhyasa/llm.html#model_label).
The list stays in memory for five minutes, because the look-up asks each
provider. Without the extra, the list has only the models in `FALLBACK`.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L129"
target="_blank" style="float:right; font-size:smaller">source</a>

### catalog

``` python
def catalog(
    ttl:float=300.0
)->list:
```

*The models that can run here, as dicts with `value`, `name`, `label`,
`provider` and `source`.*

Without the `models` extra, the list has the three Claude tiers, each
with a name.

``` python
_cat.clear()
try: import ramabana; has_rama = True
except ImportError: has_rama = False
if not has_rama: test_eq([r['name'] for r in catalog()], ['Claude Sonnet', 'Claude Haiku', 'Claude Opus'])
assert all(r['name'] for r in catalog())
```

[`Router`](https://vedicreader.github.io/abhyasa/llm.html#router) is the
`ask` the app gives the tutor. Called as it is, it uses the default
model. `for_(task)` gives the `ask` for one task: the model the settings
name for it, else the default.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L156"
target="_blank" style="float:right; font-size:smaller">source</a>

### for_task

``` python
def for_task(
    ask, task:str
):
```

*The `ask` for `task` when `ask` routes by task, else `ask` itself.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/llm.py#L142"
target="_blank" style="float:right; font-size:smaller">source</a>

### Router

``` python
def Router(
    dflt, models:NoneType=None
):
```

*The tutor’s model: a default `ask`, and one model for each task that
the settings name.*

A router with no settings uses the default for every task. A task that
names a model gets that model.

``` python
dflt = Fake(dflt='default')
r = Router(dflt, models=lambda: {})
test_eq(r('x'), 'default')
test_is(r.for_('grade'), dflt)
named = Fake(dflt='named'); named.model = 'fast'
_asks['fast'] = named
r = Router(dflt, models=lambda: dict(grade='fast'))
test_eq(r.for_('grade')('x'), 'named')
test_eq(r.for_('question')('x'), 'default')
test_is(for_task(dflt, 'grade'), dflt)
```