# ollama
``` python
from fastcore.test import test_eq, test_fail
```
## Where the daemon is
`OLLAMA_HOST` takes a bare port, a `host:port`, or a full URL. One
variable both configures the daemon and finds it, so
[`ollama_url`](https://vedicreader.github.io/rishi/ollama.html#ollama_url)
accepts all three spellings.
A loopback daemon must not go through an HTTP proxy. `httpx` reads
`$HTTPS_PROXY` from the environment by default. In a sandbox that sends
`127.0.0.1` somewhere with no route back.
[`is_local`](https://vedicreader.github.io/rishi/ollama.html#is_local)
decides whether a client trusts the environment.
------------------------------------------------------------------------
source
### is_local
``` python
def is_local(
url
):
```
*Return True when `url` points at this machine, whose daemon must be
reached without any HTTP proxy.*
------------------------------------------------------------------------
source
### ollama_url
``` python
def ollama_url(
host:NoneType=None
):
```
*Normalize an Ollama host (`:11434`, `host:port`, or a full URL) to a
base URL. `None` reads `$OLLAMA_HOST`.*
``` python
test_eq(ollama_url('http://127.0.0.1:11434'), 'http://127.0.0.1:11434')
test_eq(ollama_url('localhost:11434'), 'http://localhost:11434')
test_eq(ollama_url(':11434'), 'http://127.0.0.1:11434') # the bare-port spelling
test_eq(ollama_url('box.local'), 'http://box.local:11434') # port defaulted
test_eq(ollama_url('https://gpu.internal'), 'https://gpu.internal:443')
assert is_local(ollama_url()) and is_local(ollama_url('localhost'))
assert not is_local(ollama_url('https://gpu.internal')) # that one does go through the proxy
```
## Model ids
Ollama names models `family:tag`. It also serves any GGUF repo on the
HuggingFace Hub under an `hf.co/` prefix.
[`ollama_model`](https://vedicreader.github.io/rishi/ollama.html#ollama_model)
maps rishi’s ids onto that, with `quant` as the tag. The
`Qwen/Qwen3-4B-GGUF` constant `rishi.llama` uses runs here unchanged.
A local `.gguf` file has no address in Ollama’s store. Importing one
takes a blob upload and an `/api/create`. rishi says that rather than
failing on the wire.
------------------------------------------------------------------------
source
### ollama_model
``` python
def ollama_model(
model:NoneType=None, quant:str='Q4_K_M'
):
```
*A rishi model id as Ollama addresses it: a hub GGUF repo becomes
`hf.co/:`, an Ollama id passes through.*
``` python
test_eq(ollama_model('qwen3:4b'), 'qwen3:4b') # an Ollama id is left alone
test_eq(ollama_model('gemma3'), 'gemma3') # ...tagless too
test_eq(ollama_model(), dflt_model)
test_eq(ollama_model('Qwen/Qwen3-4B-GGUF'), 'hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M') # the rishi.llama id
test_eq(ollama_model('Qwen/Qwen3-4B-GGUF', 'Q8_0'), 'hf.co/Qwen/Qwen3-4B-GGUF:Q8_0')
test_eq(ollama_model('Qwen/Qwen3-4B-GGUF', None), 'hf.co/Qwen/Qwen3-4B-GGUF') # let Ollama choose
test_eq(ollama_model('hf.co/bartowski/x-GGUF:IQ3_M'), 'hf.co/bartowski/x-GGUF:IQ3_M')
test_fail(lambda: ollama_model('/models/mine.gguf'), contains="runtime='llama'")
```
## The wire
[`OllamaClient`](https://vedicreader.github.io/rishi/ollama.html#ollamaclient)
holds the HTTP surface rishi uses. `/api/chat` talks to a model. The
management endpoints list the local store, pull what is missing, and
name the models in memory. A refusal carries a JSON `error` field, and
[`OllamaError`](https://vedicreader.github.io/rishi/ollama.html#ollamaerror)
repeats it.
`/api/pull` and a streamed `/api/chat` both answer in newline-delimited
JSON, so one `stream` method serves both. An error can also arrive
mid-stream, after a 200. The iterator checks for that too.
------------------------------------------------------------------------
source
### OllamaClient
``` python
def OllamaClient(
host:NoneType=None, timeout:int=600, client:NoneType=None
):
```
*One Ollama daemon over HTTP: `/api/chat`, plus the model-management
endpoints rishi drives.*
------------------------------------------------------------------------
source
### OllamaError
``` python
def OllamaError(
*args, **kwargs
):
```
*What the daemon said went wrong, or that nothing answered.*
`pull` hands on Ollama’s own progress records.
[`pull_progress`](https://vedicreader.github.io/rishi/ollama.html#pull_progress)
renders them as one rewritten line, so a 4GB download looks like a
download rather than a hang.
------------------------------------------------------------------------
source
### pull_progress
``` python
def pull_progress(
out:NoneType=None
):
```
*A `pull` callback printing one rewritten line: the status, and how far
the bytes have got.*
------------------------------------------------------------------------
source
### fmt_bytes
``` python
def fmt_bytes(
n
):
```
*Bytes as a short human string.*
``` python
# the client against a scripted daemon: one `httpx.MockTransport` stands in for the whole HTTP
# surface, so requests and responses are exercised rather than mocked away
def mk_client(handler, **kw):
"An `OllamaClient` whose requests are answered by `handler` instead of a daemon."
return OllamaClient(client=httpx.Client(transport=httpx.MockTransport(handler),
base_url='http://test'), **kw)
seen = []
def handler(req):
seen.append((req.method, req.url.path, json.loads(req.content) if req.content else {}))
if req.url.path == '/api/version': return httpx.Response(200, json={'version': '0.12.0'})
if req.url.path == '/api/tags': return httpx.Response(200, json={'models': [{'name': 'qwen3:0.6b'}]})
if req.url.path == '/api/show': return httpx.Response(200, json={'capabilities': ['completion','tools','vision']})
if req.url.path == '/api/ps': return httpx.Response(200, json={'models': [{'name': 'qwen3:0.6b'}]})
if req.url.path == '/api/delete': return httpx.Response(200, text='')
return httpx.Response(404, json={'error': f'unknown path {req.url.path}'})
cl = mk_client(handler)
test_eq(cl.version(), '0.12.0')
test_eq(cl.up(), True)
test_eq(cl.models(), ['qwen3:0.6b'])
test_eq(cl.have('qwen3:0.6b'), True)
test_eq(cl.have('gemma3:1b'), False)
test_eq(cl.show('qwen3:0.6b')['capabilities'], ['completion', 'tools', 'vision'])
test_eq(len(cl.ps()), 1)
test_eq(cl.delete('qwen3:0.6b'), True)
test_eq(seen[-1][:2], ('DELETE', '/api/delete'))
test_fail(lambda: cl.get('/api/nope'), contains='unknown path') # the daemon's own words
```
``` python
import io
```
``` python
# Connection failure returns `None`; streamed errors still raise.
def dead(req): raise httpx.ConnectError('nothing listening')
test_eq(mk_client(dead).version(), None)
test_eq(mk_client(dead).up(), False)
lines = [{'status': 'pulling manifest'}, {'status': 'downloading', 'completed': 512, 'total': 1024},
{'error': 'model not found'}]
def puller(req): return httpx.Response(200, text='\n'.join(json.dumps(d) for d in lines))
got = []
test_fail(lambda: mk_client(puller).pull('nope', got.append), contains='model not found')
test_eq([d['status'] for d in got], ['pulling manifest', 'downloading']) # progress up to the failure
buf = io.StringIO()
pull_progress(buf)({'status': 'downloading', 'completed': 512*1024, 'total': 1024*1024})
assert 'downloading 512.0KB/1.0MB' in buf.getvalue()
test_eq(fmt_bytes(900), '900B')
test_eq(fmt_bytes(1536), '1.5KB')
```
## Installing Ollama
Ollama ships one release archive per platform, and nothing in it needs
root or a package manager. The published one-liner pipes into `/usr`.
The archive extracts anywhere. rishi keeps its own copy under
`~/.cache/rishi/ollama` and leaves the system alone. A machine with
`ollama` on `$PATH` keeps using that one.
Linux archives are zstd. That is stdlib from Python 3.14, and before it
`rishi[ollama]` brings `zstandard`. macOS ships gzip and Windows a zip.
------------------------------------------------------------------------
source
### ollama_bin
``` python
def ollama_bin(
dir:NoneType=None, system:bool=True
):
```
*An `ollama` binary: `$OLLAMA_BIN`, then rishi’s own install, then
`$PATH` unless `system=False`. None if there is none.*
------------------------------------------------------------------------
source
### install_url
``` python
def install_url(
system:NoneType=None, machine:NoneType=None
):
```
*URL of the Ollama release archive for this platform.*
------------------------------------------------------------------------
source
### install_ollama
``` python
def install_ollama(
dir:pathlib.PosixPath=Path('/home/runner/.cache/rishi/ollama'), url:NoneType=None, force:bool=False,
on_progress:NoneType=None
):
```
*Install Ollama under `dir` for this user: no root, no package manager,
no system service. Returns the binary.*
------------------------------------------------------------------------
source
### download
``` python
def download(
url, dest, on_progress:NoneType=None, timeout:int=600
):
```
*Stream `url` to `dest`, reporting `(done, total)` bytes to
`on_progress`.*
------------------------------------------------------------------------
source
### extract_archive
``` python
def extract_archive(
src, dest
):
```
*Extract a `.tgz`, `.tar.zst` or `.zip` release archive into `dest`.*
``` python
test_eq(install_url('Linux', 'x86_64'), 'https://ollama.com/download/ollama-linux-amd64.tar.zst')
test_eq(install_url('Linux', 'aarch64'), 'https://ollama.com/download/ollama-linux-arm64.tar.zst')
test_eq(install_url('Darwin', 'arm64'), 'https://ollama.com/download/ollama-darwin.tgz')
test_eq(install_url('Windows', 'amd64'), 'https://ollama.com/download/ollama-windows-amd64.zip')
test_fail(lambda: install_url('Plan9', 'x86_64'), contains='no Ollama release')
```
``` python
# extraction and binary discovery, on a tarball built here. `bin/ollama` is what the real Linux
# and macOS archives lay down, and what `ollama_bin` has to find without consulting $PATH
import tempfile
with tempfile.TemporaryDirectory() as d:
d = Path(d)
(d/'bin').mkdir()
(d/'bin'/'ollama').write_text('#!/bin/sh\necho fake')
with tarfile.open(d/'a.tgz', 'w:gz') as t: t.add(d/'bin'/'ollama', arcname='bin/ollama')
out = extract_archive(d/'a.tgz', d/'out')
assert (out/'bin'/'ollama').exists()
test_eq(ollama_bin(out, system=False), out/'bin'/'ollama')
test_eq(install_ollama(out), out/'bin'/'ollama') # already there, so nothing is downloaded
test_eq(ollama_bin(d/'empty', system=False), None)
# a flat archive (just `ollama`) is found too, and $OLLAMA_BIN wins over both
(d/'flat').mkdir(); (d/'flat'/'ollama').write_text('x')
test_eq(ollama_bin(d/'flat', system=False), d/'flat'/'ollama')
os.environ['OLLAMA_BIN'] = str(d/'bin'/'ollama')
try: test_eq(ollama_bin(d/'flat', system=False), d/'bin'/'ollama')
finally: del os.environ['OLLAMA_BIN']
```
## Running the daemon
[`OllamaServer`](https://vedicreader.github.io/rishi/ollama.html#ollamaserver)
starts `ollama serve` and waits for it to answer, installing Ollama
first if the machine has none.
Ollama configures some things per process rather than per request. Those
are constructor arguments here. They cover the model store, the default
context length, how long a model stays resident, flash attention, KV
cache quantization, and the parallel and loaded-model limits.
A daemon already listening is left alone. Someone else’s server is not
rishi’s to restart or reconfigure. One rishi starts is stopped at
interpreter exit, so a notebook leaks nothing. It gets `/dev/null` as
stdin, so the daemon never shares the caller’s terminal.
------------------------------------------------------------------------
source
### OllamaServer
``` python
def OllamaServer(
host:NoneType=None, bin:NoneType=None, dir:pathlib.PosixPath=Path('/home/runner/.cache/rishi/ollama'),
models:NoneType=None, n_ctx:NoneType=None, keep_alive:NoneType=None, flash_attn:NoneType=None,
kv_cache_type:NoneType=None, num_parallel:NoneType=None, max_loaded:NoneType=None, log:NoneType=None
):
```
*An `ollama serve` process rishi starts and owns, installing Ollama
first if the machine has none.*
------------------------------------------------------------------------
source
### stop_ollama
``` python
def stop_ollama(
host:NoneType=None
):
```
*Stop the daemon rishi started at `host`, or every one of them when
`host` is None.*
------------------------------------------------------------------------
source
### ensure_ollama
``` python
def ensure_ollama(
host:NoneType=None, install:bool=True, serve:bool=True, timeout:int=60, on_progress:NoneType=None,
srv_kw:NoneType=None
):
```
*A client for a live daemon at `host`: reuse one already listening, else
start one, installing Ollama if needed.*
``` python
# the process-level configuration, which is the half of Ollama that is not a request field
s = OllamaServer('localhost:11500', models='/data/models', n_ctx=8192, keep_alive='30m',
flash_attn=True, kv_cache_type='q8_0', num_parallel=2)
e = s.env()
test_eq(e['OLLAMA_HOST'], 'http://localhost:11500')
test_eq(e['OLLAMA_MODELS'], '/data/models')
test_eq(e['OLLAMA_CONTEXT_LENGTH'], '8192')
test_eq(e['OLLAMA_KEEP_ALIVE'], '30m')
test_eq(e['OLLAMA_FLASH_ATTENTION'], '1') # a bool, in the spelling Ollama reads
test_eq(e['OLLAMA_KV_CACHE_TYPE'], 'q8_0')
test_eq(e['OLLAMA_NUM_PARALLEL'], '2')
assert 'OLLAMA_MAX_LOADED_MODELS' not in e # unset stays unset, rather than guessing a default
test_eq(s.running, False)
test_eq(s.stop().running, False) # stopping one that never started is fine
```
``` python
# with nothing listening and no leave to start anything, the error names both ways out
test_fail(lambda: ensure_ollama('127.0.0.1:11599', serve=False), contains='ollama serve')
test_fail(lambda: OllamaServer('127.0.0.1:11599', bin='/nonexistent/ollama').start(install=False, timeout=1),
contains='no ollama binary')
# the daemon gets end of file as stdin, so the caller's terminal keeps its waiting line and stays blocking
import os, pty, subprocess, sys, tempfile
from pathlib import Path
fake = Path(tempfile.mkdtemp())/'ollama'
fake.write_text(f'#!{sys.executable}\nimport os, sys; t = os.isatty(0); os.set_blocking(0, False); open(sys.argv[0] + ".tty", "w").write(str(t)); sys.exit(3)\n'); fake.chmod(0o755)
m, s = pty.openpty(); os.write(m, b'x\n')
probe = f"""import os, select
from rishi.ollama import OllamaServer
try: OllamaServer('127.0.0.1:11599', bin={str(fake)!r}).start(install=False, timeout=5)
except Exception as e: print(type(e).__name__)
print(open({str(fake)!r} + '.tty').read(), os.get_blocking(0), os.read(0, 9) if select.select([0], [], [], 1)[0] else b'')"""
got = subprocess.run([sys.executable, '-c', probe], stdin=s, capture_output=True, text=True, timeout=60).stdout.split()
os.close(s); os.close(m)
test_eq(got, ['OllamaError', 'False', 'True', "b'x\\n'"])
```
## Messages
Ollama’s message shape is OpenAI’s, with two differences. Images sit in
a sibling `images` list of bare base64 rather than in content parts.
Thinking has its own `thinking` field rather than `` tags in the
text.
[`to_ollama_msg`](https://vedicreader.github.io/rishi/ollama.html#to_ollama_msg)
converts rishi’s canonical history to that, and carries assistant
thinking back out so a resumed conversation keeps it.
Images go out on every turn rather than collapsing to a placeholder, as
on `remote` and unlike `llama`. The daemon holds the KV cache, so a
repeat costs a prefix match rather than a re-encode.
Audio has nowhere to go here. `rishi.llama` (mtmd) and `rishi.mlx` both
take it, and the error says so rather than dropping the clip.
------------------------------------------------------------------------
source
### to_ollama_msgs
``` python
def to_ollama_msgs(
msgs, sp:str=''
):
```
*Convert rishi history to Ollama messages, with `sp` as the leading
system message.*
------------------------------------------------------------------------
source
### to_ollama_msg
``` python
def to_ollama_msg(
m
):
```
*Convert one rishi history message to an Ollama chat message.*
A reply carries thinking in a `thinking` field and calls in
`tool_calls`. A model whose chat template was built for neither writes
`` and `` into the text instead.
[`norm_ochat`](https://vedicreader.github.io/rishi/ollama.html#norm_ochat)
reads both forms, as Urai’s `norm_resp` does for llama.cpp, so an
arbitrary GGUF from the Hub behaves like a model from Ollama’s own
library.
Usage comes from Ollama’s eval counts. Ollama reports no figure for
reuse, so `use.cached_tokens` reads zero even when the daemon is reusing
the conversation’s prefix.
------------------------------------------------------------------------
source
### norm_ochat
``` python
def norm_ochat(
r, model:NoneType=None
):
```
*Convert one `/api/chat` response to a rishi `Resp`, reading ``
and `` text as well as the native fields.*
------------------------------------------------------------------------
source
### ollama_usage
``` python
def ollama_usage(
r, model:NoneType=None
):
```
*Convert Ollama’s eval counts to a rishi usage dict.*
``` python
# text, images and thinking, each into the field Ollama reads it from
test_eq(to_ollama_msg({'role': 'user', 'content': 'hi'}), {'role': 'user', 'content': 'hi'})
m = to_ollama_msg(mk_msg(['what is this?', Path('images.jpeg').read_bytes()]))
test_eq(m['content'], 'what is this?')
test_eq(len(m['images']), 1)
assert not m['images'][0].startswith('data:') # bare base64, not a data URL
test_eq(to_ollama_msg({'role': 'assistant', 'content': 'a', 'channels': {'thought': 't'}})['thinking'], 't')
# a tool call goes out as an object, and its result comes back named
call_msg = {'role': 'assistant', 'content': '', 'tool_calls': [ToolCall(name='add', arguments={'a': 1, 'b': 2})]}
test_eq(to_ollama_msg(call_msg)['tool_calls'], [{'function': {'name': 'add', 'arguments': {'a': 1, 'b': 2}}}])
test_eq(to_ollama_msg({'role': 'tool', 'name': 'add', 'content': '3'}),
{'role': 'tool', 'content': '3', 'tool_name': 'add'})
# the system prompt leads, and a system message already in `hist` is dropped rather than duplicated
ms = to_ollama_msgs([{'role': 'system', 'content': 'old'}, {'role': 'user', 'content': 'hi'}], sp='be brief')
test_eq(ms, [{'role': 'system', 'content': 'be brief'}, {'role': 'user', 'content': 'hi'}])
test_fail(lambda: to_ollama_msg(mk_msg([Path('speech.wav').read_bytes(), 'transcribe'])),
contains="runtime='llama'")
```
``` python
# the native fields
r = norm_ochat({'model': 'qwen3:0.6b', 'done_reason': 'stop',
'message': {'role': 'assistant', 'content': 'hello', 'thinking': 'thinking hard'},
'prompt_eval_count': 10, 'eval_count': 4})
test_eq(resp_text(r), 'hello')
test_eq(thought(r), 'thinking hard')
test_eq(r['usage'], {'prompt_tokens': 10, 'completion_tokens': 4, 'total_tokens': 14, 'model': 'qwen3:0.6b'})
assert 'truncated' not in r
test_eq(truncated(norm_ochat({'message': {'content': 'x'}, 'done_reason': 'length'})), True)
# a native tool call: arguments as an object, and an id invented because Ollama sends none
r = norm_ochat({'message': {'tool_calls': [{'function': {'name': 'add', 'arguments': {'a': 1}}}]}})
test_eq(len(r['tool_calls']), 1)
test_eq(r['tool_calls'][0].name, 'add')
test_eq(r['tool_calls'][0].arguments, {'a': 1})
assert r['tool_calls'][0]['id']
# and a model that writes tags into the text instead, as an arbitrary GGUF off the Hub will
r = norm_ochat({'message': {'content': 'hmmok\n{"name": "add", "arguments": {"a": 1}}'}})
test_eq(resp_text(r), 'ok')
test_eq(thought(r), 'hmm')
test_eq(r['tool_calls'][0].name, 'add')
```
## OllamaChat
The tool loop, approval, budget, callbacks and context handling come
from Urai, as on every backend.
[`OllamaChat`](https://vedicreader.github.io/rishi/ollama.html#ollamachat)
adds the request and the daemon it goes to.
`think` is a field of the request here, not a switch in the system
prompt. It takes `True`, `False`, or a level string such as `'low'` or
`'high'`, which is sent unchanged. A model that cannot think refuses the
field; the chat then retries once without it and stops sending it.
The context window comes from the portable `ctx` option, which also
accepts the `rishi.llama` name `n_ctx`, and goes out as Ollama’s
`num_ctx`. `n_gpu_layers` goes out as `num_gpu`. Anything else Ollama’s
sampler takes goes in `options=`. `structured` uses Ollama’s `format`
field, so the reply is constrained to the schema instead of merely asked
for it. With `pull=True`, the default, a missing model is downloaded
when the chat is built, as the llama backend downloads a missing GGUF.
`keep_alive` says how long the daemon keeps the model loaded after a
turn, and `unload()` drops it at once.
`ctx_limit` is the window the chat believes it has. It follows `ctx`
when you set one, else the daemon’s own default, which is
`$OLLAMA_CONTEXT_LENGTH` or 4096 tokens whatever the model was trained
for. Urai’s `pct_full` and `SlidingWindowCallback` depend on it being
right.
------------------------------------------------------------------------
source
### OllamaChat
``` python
def OllamaChat(
model:NoneType=None, *, runtime:NoneType=None, model_path:NoneType=None, opts:NoneType=None, host:NoneType=None,
client:NoneType=None, # a daemon to use instead of finding or starting one
quant:str='Q4_K_M', # which quantization to name when pulling
pull:bool=True, install:bool=True, serve:bool=True, srv_kw:NoneType=None, on_progress:NoneType=None,
n_gpu_layers:NoneType=None, keep_alive:NoneType=None, # how long the daemon holds the weights after a turn
options:NoneType=None, # Ollama's own `options` block, merged last
comp_kw:NoneType=None, # merged into the request body verbatim
**kw
):
```
*A `Chat` that sends each turn to an Ollama daemon, starting the daemon
and pulling the model when needed.*
------------------------------------------------------------------------
source
### dflt_ctx
``` python
def dflt_ctx():
```
*The context length the daemon uses when a request names none:
`$OLLAMA_CONTEXT_LENGTH`, else Ollama’s own 4096.*
``` python
from dataclasses import dataclass
from typing import Literal
@dataclass
class Pick:
"A choice."
a: Literal['x', 'y']
c, seen = OllamaChat.__new__(OllamaChat), {}
c.model_id = 'm'
c._payload = lambda **kw: seen.update(kw) or kw
c._ask = lambda p: {'message': {'role': 'assistant', 'content': '{"a": "x"}'}, 'done': True}
test_eq(c._structured_call('q', Pick, ''), {'a': 'x'})
test_eq(seen['fmt']['properties']['a']['enum'], ['x', 'y']) # the schema goes through urai's mk_toolspec, which spells a Literal as an enum
```
`/api/show` reports what a model can do, and it is more reliable than
any guess Urai’s `model_caps` makes elsewhere, because the daemon has
the weights. `model_caps(model, runtime='ollama')` asks it through
[`ollama_caps`](https://vedicreader.github.io/rishi/ollama.html#ollama_caps),
which maps Ollama’s `vision` capability to image input. When no daemon
answers, or the model is not in its store, the answer falls back to text
only.
------------------------------------------------------------------------
source
### ollama_caps
``` python
def ollama_caps(
model:NoneType=None, client:NoneType=None, quant:str='Q4_K_M'
):
```
*What an Ollama model accepts, from the daemon’s own `/api/show`. None
when no daemon answers.*
### Against a real daemon
Everything above runs offline, against a scripted daemon. The cells
below need Ollama and download a model on first use, so they are marked
`#| eval: false` and the test run skips them. On a bare machine,
[`OllamaChat`](https://vedicreader.github.io/rishi/ollama.html#ollamachat)
installs Ollama under `~/.cache/rishi/ollama`, starts it, pulls the
model and answers.
In the saved output of the last cell, `gemma3:4b` reports text input
only, with source `default`. That model was not in the daemon’s store,
so `/api/show` failed and the capabilities fell back to Urai’s default.
``` python
chat = Chat('ollama/qwen3:0.6b', think=False, sp='You are concise.')
print(resp_text(chat('Give me one fact about lobsters.')))
print(chat.use)
```
Lobsters are marine animals with a soft, flexible body and a powerful, claw-like appendage.
total=55|in=33|out=22|turns=1|model=qwen3:0.6b
``` python
# thinking levels, tools and structured output, on the real thing
def add(a: int, b: int) -> int:
"Add two integers."
return a + b
from dataclasses import dataclass
@dataclass
class Person: name:str; age:int
t = Chat('ollama/qwen3:0.6b', tools=[add], think='low')
print(resp_text(t('What is 2 + 3? Use the add tool.')))
print(t.structured('Extract: John Smith is 30.', Person))
print(t.classify('I loved this film!', ['positive', 'negative']))
t.close()
```
Person(name='John Smith', age=30)
positive
``` python
# the daemon, driven from Python: what is downloaded, what is loaded, what a model can do
cl = ensure_ollama()
print(cl.version(), cl.models())
print([m['name'] for m in cl.ps()])
print(model_caps('qwen3:0.6b', runtime='ollama'))
print(model_caps('gemma3:4b', runtime='ollama')) # vision, per the daemon itself
```
0.32.15 ['ornith-1.5:9b', 'qwen3:0.6b', 'qwen3.6:27b-coding-nvfp4']
['qwen3:0.6b']
Caps(in=text, out=text, runtime)
Caps(in=text, out=text, default)
## What this backend does not do
Ollama’s chat API carries images but not audio; `rishi.llama` and
`rishi.mlx` take audio. There is no saved KV cache either. `rishi.llama`
and `rishi.mlx` write theirs to disk and read it back, but Ollama keeps
its cache inside the daemon and offers no handle on it, so
`use.cached_tokens` reads zero even while the daemon reuses the prefix.
Token counts before a turn are estimates, because Ollama has no
tokenizer endpoint; the real count arrives with the reply, in `use`.
Local `.gguf` paths are refused, because Ollama serves only what is in
its own store, and `runtime='llama'` is the backend for a file. There is
no broker.
[`LlamaBroker`](https://vedicreader.github.io/rishi/llama.html#llamabroker)
exists because one `Llama` belongs to one process, while the Ollama
daemon already serves many clients; size it with `OLLAMA_NUM_PARALLEL`
and `OLLAMA_MAX_LOADED_MODELS`.
### Tests
The tests on this page cover host parsing, model ids, the HTTP client
against a scripted daemon (`httpx.MockTransport`), the installer, the
server’s environment, message conversion and response normalisation.
None of them needs Ollama, so the test run executes all of them.
[`OllamaChat`](https://vedicreader.github.io/rishi/ollama.html#ollamachat)
itself runs only in the manual cells above.