# tutor
This module runs the learning loop.
[`pick`](https://vedicreader.github.io/abhyasa/model.html#pick) chooses
the concept and the depth, and a model writes the question. The learner
answers and can ask for hints. The model grades the answer, and the
learner model updates the concept. After a miss, the tutor teaches the
smallest step. Each surface (the web app, the ramabana terminal agent, a
notebook) uses
[`Tutor`](https://vedicreader.github.io/abhyasa/tutor.html#tutor) and
[`Session`](https://vedicreader.github.io/abhyasa/tutor.html#session).
Import the tools that this module uses, and the other abhyasa modules
that the tutor calls.
## Tries and hints
A wrong answer does not end the question. The tutor names the first
error, gives no fix, and the learner tries again. After `MAX_TRIES`
wrong tries, or when the learner stops, the tutor teaches the smallest
step. Then the state of the concept records the miss. A pass after a
wrong try counts as `fragile`.
Each open question keeps a thread, `q.turns`: the question, then each
message of the learner and each reply of the tutor, in order. A message
of the learner is only what they say this time, often a follow-up to the
last nudge. They do not send the full answer again. The grade, the hints
and the lesson read the whole thread.
`HINTS` is the hint ladder, from the least to the most revealing. The
model gets one rung for each hint request.
`DEPTH_ASK` tells the model what a question at each depth must make the
learner do.
`TUTOR_SP` is the system prompt for questions, hints and lessons.
## Prompts
Each prompt function makes the text for one model call. The prompt text
is an instruction to the model.
[`_concept_block`](https://vedicreader.github.io/abhyasa/tutor.html#_concept_block)
describes the concept and the learner.
[`question_prompt`](https://vedicreader.github.io/abhyasa/tutor.html#question_prompt)
asks for one question at one depth. `who` is the profile of the learner
as text (see `profile`), so the question uses their world.
------------------------------------------------------------------------
source
### question_prompt
``` python
def question_prompt(
c, depth, avoid:tuple=(), ctx:str='', who:str='', bridge:tuple=(), method:str=''
):
```
*The prompt that writes one question on `c` at `depth` in `method`;
`bridge` names known ideas that a transfer question can connect to.*
A transfer question asks the learner to use an idea in a new place. The
best new place is one that they already know.
[`bridges`](https://vedicreader.github.io/abhyasa/tutor.html#bridges)
names up to three concepts that the learner knows to depth 3 or more:
from other subjects first, then linked ones, then the deepest. Thus a
transfer question on attention can use a Sanskrit or jyotiṣa idea.
------------------------------------------------------------------------
source
### bridges
``` python
def bridges(
c, concepts, n:int=3
)->list:
```
*The names of up to `n` concepts known to depth 3 or more, for a
transfer question on `c`: other subjects first, then linked ones.*
[`thread_text`](https://vedicreader.github.io/abhyasa/tutor.html#thread_text)
writes the thread as plain lines for a prompt. `GRADE_SP` and
[`grade_prompt`](https://vedicreader.github.io/abhyasa/tutor.html#grade_prompt)
grade the conversation against the hidden model answer and the rubric.
The latest message comes last, with the confidence that the learner gave
it.
------------------------------------------------------------------------
source
### grade_prompt
``` python
def grade_prompt(
q, answer, thinking:str='', confidence:NoneType=None
):
```
*The prompt that grades the conversation on question `q`, with
[`answer`](https://vedicreader.github.io/abhyasa/rama.html#answer) as
the latest message of the learner.*
------------------------------------------------------------------------
source
### thread_text
``` python
def thread_text(
q
)->str:
```
*The conversation on `q` so far, one turn to a line, in order.*
[`hint_prompt`](https://vedicreader.github.io/abhyasa/tutor.html#hint_prompt)
asks for one rung of the hint ladder. It reads the thread, so the hint
fits what the learner said.
------------------------------------------------------------------------
source
### hint_prompt
``` python
def hint_prompt(
q, level
):
```
*The prompt for hint `level` (0-5) on question `q`.*
[`teach_prompt`](https://vedicreader.github.io/abhyasa/tutor.html#teach_prompt)
asks for the lesson after a miss. The lesson ends with a check question.
------------------------------------------------------------------------
source
### teach_prompt
``` python
def teach_prompt(
q, first_error
):
```
*The prompt that teaches the smallest step after a miss.*
## The tutor
[`Tutor`](https://vedicreader.github.io/abhyasa/tutor.html#tutor) keeps
the open questions and the check questions from lessons.
[`_ready_q`](https://vedicreader.github.io/abhyasa/tutor.html#_ready_q)
finds a question that came with the source of a concept. After each
`observe_every` closed questions,
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
(from `profile`) reads the recent answers in the background and updates
the profile.
------------------------------------------------------------------------
source
### Tutor
``` python
def Tutor(
learning:abhyasa.store.Learning, ask, goal:int=4, context:bool=True, sources:NoneType=None, pool:NoneType=None,
observe_every:int=6
):
```
*The learning loop over a
[`Learning`](https://vedicreader.github.io/abhyasa/store.html#learning)
store with an `ask(prompt, sp)` model; `pool` runs background jobs.*
## Ask a question
`ask_on` uses the first source that it finds: the check from the last
lesson, a ready question, or a new question from the model. It shuffles
the options of a choice question, and starts the thread with the
question.
[`public`](https://vedicreader.github.io/abhyasa/tutor.html#public)
removes the fields that the learner must not see.
------------------------------------------------------------------------
source
### public
``` python
def public(
q
)->fastcore.basics.AttrDict:
```
*A question as the learner can see it: no model answer, no rubric, no
correct option, no misconception tags.*
------------------------------------------------------------------------
source
### Tutor.ask_on
``` python
def ask_on(
c, depth:int, why:str='', session:str='', mode:str=None
)->fastcore.basics.AttrDict:
```
*Write a question on concept `c` at `depth` in a method from
[`choose`](https://vedicreader.github.io/abhyasa/method.html#choose),
keep it open, and return it without its answer fields. `mode='drill'`
uses your best method.*
`next` lets
[`pick`](https://vedicreader.github.io/abhyasa/model.html#pick) choose
the concept, then asks a question on it.
[`paced`](https://vedicreader.github.io/abhyasa/tutor.html#paced) moves
the depth for the pace in the profile of the learner.
------------------------------------------------------------------------
source
### Tutor.next
``` python
def next(
focus:str=None, subject:str=None, session:str='', skip:tuple=(), near:tuple=(), pace:str=None, mode:str=None
)->fastcore.basics.AttrDict | None:
```
*The next question, chosen by
[`pick`](https://vedicreader.github.io/abhyasa/model.html#pick) at
`pace` (by default the pace in the profile), or None when there is
nothing to learn or review.*
------------------------------------------------------------------------
source
### paced
``` python
def paced(
c, depth:int, pace:str='steady'
)->int:
```
*The depth for the pace of the learner: at `gentle`, at most one above
the floor; at `fast`, one more than
[`pick`](https://vedicreader.github.io/abhyasa/model.html#pick) gives.*
[`flow`](https://vedicreader.github.io/abhyasa/tutor.html#flow) keeps a
session in its band, at about 85% success. It reads the last outcomes of
the session. After two misses in a row, the next question is one pace
easier. After three solid passes in a row, it is one pace harder. The
note says why, so the learner sees the change.
------------------------------------------------------------------------
source
### flow
``` python
def flow(
log, pace:str='steady', n:int=3
)->tuple:
```
*The `(pace, note)` for the next question of a session: one pace easier
after two misses, one pace harder after `n` solid passes.*
## Hints
Each call to
[`hint`](https://vedicreader.github.io/abhyasa/rama.html#hint) gives the
next rung of the ladder. After the last rung, it gives the last hint
again.
------------------------------------------------------------------------
source
### Tutor.hint
``` python
def hint(
qid:str
)->str:
```
*The next hint on the ladder, from the least to the most revealing.*
## Grade an answer
`_grade` grades a choice question in code and other questions with the
model.
[`answer`](https://vedicreader.github.io/abhyasa/rama.html#answer) adds
the message of the learner to the thread, and the nudge after a wrong
try. A pass or the last try closes the question.
------------------------------------------------------------------------
source
### Tutor.answer
``` python
def answer(
qid:str, answer:str='', thinking:str='', confidence:int=None, dont_know:bool=False
)->fastcore.basics.AttrDict:
```
*Grade an answer. A wrong try keeps the question open with a nudge. A
pass or the last try closes it and updates the concept.*
## Stop and teach
`give_up` closes the question as a miss. `_close` updates the concept,
teaches after a miss and records the attempt. Then each function in
`CLOSE_HOOKS` can add to the result. The demo module uses this hook to
attach a demo after a miss. The confidence of the first try counts,
because the first try shows what the learner believed.
------------------------------------------------------------------------
source
### Tutor.give_up
``` python
def give_up(
qid:str, confidence:int=0, dont_know:bool=False
)->fastcore.basics.AttrDict:
```
*Close the question as a miss and teach the smallest step. A shown
answer always counts as a miss.*
`_lesson` writes the lesson after a miss. When the model fails, the
lesson is a note with the model answer.
The tutor keeps the last 20 closed questions in `shut`.
[`teach`](https://vedicreader.github.io/abhyasa/rama.html#teach) gives a
lesson on one of them when you ask for it, for example after a shaky
pass. It does not change the learner state. A second call gives the same
lesson.
------------------------------------------------------------------------
source
### Tutor.teach
``` python
def teach(
qid:str, another:bool=False
)->str:
```
*A lesson on request for the closed question `qid`, kept in its thread
and on its attempt. `another` uses a different method. The learner state
does not change.*
## A session
[`Session`](https://vedicreader.github.io/abhyasa/tutor.html#session)
keeps one sitting together. Its id goes on each attempt, and it stops
after `limit` questions (by default, the session length in the profile
of the learner). It asks about one concept for at most `run` questions
in a row. This lets the range between floor and ceiling close, and it
keeps the learner interested.
A session is saved on the shelf after each move: when a question opens,
after each answer and hint, and at the end. It keeps the log of each
closed question (with the floor before and after, and the thread) and
the open question with its thread. Thus
[`Session.load`](https://vedicreader.github.io/abhyasa/tutor.html#session.load)
continues it after a restart, with the same question, graded the same
way.
A session in a journey asks about the current step of the journey. When
the step is learned, the next question is on the next step.
------------------------------------------------------------------------
source
### Session
``` python
def Session(
tutor:__main__.Tutor, focus:str=None, subject:str=None, limit:int=None, run:int=3, journey:str=None,
sid:str=None, drill:bool=False
):
```
*One sitting: questions and answers, saved on the shelf after each move,
so that a restart can continue it.*
A check right after a lesson shows that you understood it. It does not
show that you will remember it, because the lesson is still in your
mind. Thus each concept that you miss in a session comes back one more
time before the end. At least two other questions come between the miss
and the retest.
[`retests`](https://vedicreader.github.io/abhyasa/tutor.html#retests)
finds these concepts. A pass right after the miss does not count.
------------------------------------------------------------------------
source
### retests
``` python
def retests(
log, gap:int=2
)->dict:
```
*The concepts missed in a session log, with no pass at least `gap`
questions after the miss: concept -\> its last miss, oldest first.*
`_retest` asks one of them when only enough questions are left to retest
each miss once.
A drill session (“Practice my weak spots”) asks your weakest concepts
first, each one time, with your best method. When none is left, the
session goes on as usual.
[`answer`](https://vedicreader.github.io/abhyasa/rama.html#answer),
`give_up` and
[`hint`](https://vedicreader.github.io/abhyasa/rama.html#hint) act on
the open question and save the session. A closed question goes in the
log with what moved.
------------------------------------------------------------------------
source
### Session.hint
``` python
def hint()->str:
```
------------------------------------------------------------------------
source
### Session.give_up
``` python
def give_up(
**kw
)->fastcore.basics.AttrDict:
```
------------------------------------------------------------------------
source
### Session.answer
``` python
def answer(
*args, **kw
)->fastcore.basics.AttrDict:
```
`save` writes the session to the shelf. `summary` gives one line. `end`
closes the session. `load` reads a session back, with its open question.
------------------------------------------------------------------------
source
### Session.load
``` python
def load(
tutor:__main__.Tutor, sid:str
):
```
*The session `sid` from the shelf, with its open question open again;
None for an unknown id or an old summary.*
------------------------------------------------------------------------
source
### Session.end
``` python
def end()->str:
```
*Close the session and file it. Return its summary.*
------------------------------------------------------------------------
source
### Session.save
``` python
def save():
```
*File the session on the shelf, with its log and its open question. A
session with nothing in it is not filed.*
------------------------------------------------------------------------
source
### Session.summary
``` python
def summary()->str:
```
*A short summary, and the text of the session document.*
[`Session.teach`](https://vedicreader.github.io/abhyasa/tutor.html#session.teach)
asks for that lesson on a question of the session, and keeps it in the
log, so the thread shows it after a restart.
------------------------------------------------------------------------
source
### Session.teach
``` python
def teach(
qid:str, another:bool=False
)->str:
```
*A lesson on request for `qid`, a closed question of this session; the
log keeps it.*
[`moved`](https://vedicreader.github.io/abhyasa/tutor.html#moved) says
what a session changed: for each concept, its floor before the first
question and after the last one.
------------------------------------------------------------------------
source
### moved
``` python
def moved(
log
)->list:
```
*For each concept in a session log, in order: its name, the floor before
and after, and the misconceptions found.*
## Tests
A fake model plays the tutor. Its grader passes an answer only when the
answer has the words “rank position”. Two concepts are on the shelf, and
`Reciprocal rank fusion` needs `Ranking`.
``` python
from abhyasa.store import temp_learning
from abhyasa.llm import Fake
lr = temp_learning()
lr.put('Ranking', 'Order items by a score.', 'ml', order=0)
lr.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank).', 'ml', prereqs=['ranking'], order=1)
def grader(p, sp): return dict(correct='rank position' in p.split("Learner's answer:")[1], grade=7, first_error='You add raw scores.',
misconception='scores are comparable across systems', feedback='Look at what each list really shares.')
fake = Fake({'You grade one answer': grader, 'Give hint level': 'Look at what the two lists have in common.',
'Teach the smallest step': 'Only positions are shared. Check: what happens with k = 0?',
'Write ONE question': dict(question='Why does RRF use ranks, not scores?', answer='Scores are not comparable.', rubric=['ranks are comparable'])})
t = Tutor(lr, fake, context=False)
```
/opt/hostedtoolcache/Python/3.12.15/x64/lib/python3.12/site-packages/usearch/__init__.py:131: UserWarning: Will download `usearch_sqlite` binary from GitHub.
warnings.warn("Will download `usearch_sqlite` binary from GitHub.", UserWarning)
The first question is on the prereq, at depth 3. The learner does not
see the answer or the rubric.
``` python
q = t.next()
test_eq((q.concept, q.depth, q.kind), ('ranking', 3, 'explain')) # rrf waits for its prereq
assert 'answer' not in q and 'rubric' not in q
```
A wrong answer keeps the question open and names the first error. Then a
hint comes from the ladder.
``` python
r = t.answer(q.id, 'sum the scores', confidence=90)
test_eq((r.done, r.correct, r.tries_left), (False, False, 2))
test_eq(r.first_error, 'You add raw scores.')
test_eq(t.hint(q.id), 'Look at what the two lists have in common.')
```
A pass after a wrong try and a hint is `fragile`. The floor stays, and
the concept is `shaky` at depth 3.
``` python
r = t.answer(q.id, 'use the rank position', thinking='positions are shared', confidence=90)
test_eq((r.done, r.correct, r.outcome), (True, True, 'fragile')) # passed, but after a wrong try and a hint
c = lr.concept('ranking')
test_eq((c.floor, c.shaky, c.seen), (0, 3, 1))
test_eq(len(lr.attempts('ranking')), 2)
```
The thread holds each turn in order: the question, the wrong try, its
nudge, the hint and the pass.
``` python
test_eq([(x['who'], x.get('kind')) for x in q.turns],
[('tutor', 'question'), ('you', None), ('tutor', 'nudge'), ('tutor', 'hint'), ('you', None), ('tutor', 'close')])
test_eq(lr.attempts('ranking')[0].thread[1]['text'], 'sum the scores') # the final attempt keeps the thread
```
Three wrong tries close the question, keep the misconception and teach.
``` python
q = t.next(focus='ranking')
test_eq(q.depth, 3) # shaky at explain: ask at explain again
for _ in range(3): r = t.answer(q.id, 'sum the scores', confidence=85)
test_eq((r.done, r.outcome), (True, 'misconception'))
assert 'Check:' in r.lesson
c = lr.concept('ranking')
test_eq((c.ceiling, c.misconceptions), (3, ['scores are comparable across systems']))
```
When the lesson model fails, the question still closes, the miss counts
once, and the lesson says what happened.
``` python
fl = temp_learning(); fl.put('Ranking', 'Order items by a score.', 'ml')
def broken(p, sp=''):
if 'Teach the smallest step' in p: raise RuntimeError('the model is down')
return fake(p, sp)
ft = Tutor(fl, broken, context=False)
q = ft.next()
for a in ('sum the scores', 'add the scores', 'average the scores'): r = ft.answer(q.id, a, confidence=85)
test_eq((r.done, q.id in ft.open), (True, False))
assert 'could not write the lesson' in r.lesson and 'Scores are not comparable' in r.lesson
c = fl.concept('ranking')
test_eq((c.seen, c.misses), (1, 1))
test_eq(len([a for a in fl.attempts('ranking') if a.get('final')]), 1)
```
The check question from the lesson comes next.
``` python
q = t.next(focus='ranking')
test_eq(q.question, 'what happens with k = 0?') # the lesson's check comes next
t.open.clear()
```
A ready question from a source comes before a question from the model.
``` python
lr.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', questions=[dict(q='Join na + asti.', a='nāsti', depth=3)])
q = t.next(subject='sanskrit')
test_eq(q.question, 'Join na + asti.')
```
Code grades a choice question. A wrong option gives the misconception
that it matches.
``` python
fake.rules['Write ONE question'] = dict(question='Which is a sandhi?', options=[dict(text='na + asti → nāsti', misconception=''),
dict(text='na + asti → naasti', misconception='vowels just sit side by side')], correct=0, answer='', rubric=[])
t.open.clear()
lr.update('sandhi', floor=0, ceiling=2)
q = t.next(subject='sanskrit')
test_eq(q.kind, 'choice')
r = t.answer(q.id, q.options.index('na + asti → naasti'), confidence=90)
test_eq(r.first_error, 'This option assumes: vowels just sit side by side')
```
A session stops after `limit` questions and files a summary.
``` python
fake2 = Fake({k: fake.rules[k] for k in ('You grade one answer', 'Teach the smallest step')},
dflt=dict(question='What does a rank tell you?', answer='Its position.', rubric=['position']))
s = Session(Tutor(temp_learning(), fake2, context=False), limit=2)
s.tutor.learning.put('Ranking', 'Order items by a score.', 'ml')
assert s.next() is not None
s.give_up()
assert s.next() is not None
test_eq(s.answer('the rank position', confidence=80).correct, True)
test_eq(s.next(), None)
test_eq(s.summary(), '1 of 2 passed. Concepts: ranking.')
test_eq(s.tutor.learning.session(s.id).asked, 2) # filed as it goes, not only at the end
```
A session on one concept stays in its subject. It asks about the
neighbors of the concept first.
``` python
sl = temp_learning()
sl.put('Attention', 'Weights from query-key dot products.', 'ml', order=0)
sl.put('Softmax', 'Turns scores into weights that sum to 1.', 'ml', related=['attention'], order=5)
sl.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', order=0)
ss = Session(Tutor(sl, fake2, context=False), focus='attention')
test_eq((ss.subject, ss.near), ('ml', {'softmax'}))
sl.update('attention', floor=4, seen=1, due=1e12) # learned, and not due: the session moves on, and stays in ml
test_eq(ss.next().concept, 'softmax')
```
A session is saved with its open question. `load` continues it in a new
tutor, as after a restart, and grades the same question.
``` python
pl = temp_learning(); pl.put('Ranking', 'Order items by a score.', 'ml')
ps = Session(Tutor(pl, fake2, context=False), limit=3)
q1 = ps.next(); ps.give_up()
q2 = ps.next(); ps.answer('a score', confidence=60) # a wrong try: the question stays open
t2 = Tutor(pl, fake2, context=False) # a restart: a new tutor with nothing open
back = Session.load(t2, ps.id)
test_eq((back.q.id, back.q.question, len(back.log), back.limit), (q2.id, q2.question, 1, 3))
assert 'answer' not in back.q and back.q.id in t2.open # the learner view, and the full question in the tutor
test_eq(back.answer('the rank position', confidence=80).correct, True)
test_eq([(r['outcome'], r['floor0']) for r in back.log], [('gap', 0), ('fragile', 0)])
test_eq([t['who'] for t in back.log[-1]['turns']], ['tutor', 'you', 'tutor', 'you', 'tutor'])
```
[`moved`](https://vedicreader.github.io/abhyasa/tutor.html#moved) gives
the floor before and after for each concept of the log.
``` python
test_eq([(m['concept'], m['before'], m['after'], m['asked'], m['passed']) for m in moved(back.log)], [('ranking', 0, 0, 2, 1)])
test_eq(moved([dict(concept='a', name='A', floor0=2, floor=3, correct=True), dict(concept='a', floor0=3, floor=4, correct=True, misconception='x')]),
[dict(concept='a', name='A', before=2, after=4, asked=2, passed=2, misconceptions=['x'])])
```
`end` closes the session. An ended session loads with no open question
and gives no more questions.
``` python
back.next(); back.end()
again = Session.load(Tutor(pl, fake2, context=False), ps.id)
test_eq((again.q, again.next(), bool(again.ended)), (None, None, True))
test_is(Session.load(t2, 'nope'), None)
```
A session in a journey asks about the current step. When the step is
learned, it goes to the next step. It stops at the end of the journey.
``` python
jl = temp_learning()
for n in ('Tokenizers', 'Embeddings', 'Sandhi'): jl.put(n, f'{n} statement.', 'ml' if n != 'Sandhi' else 'sanskrit')
jl.update('sandhi', seen=1, due=1.) # a review due elsewhere does not break in
j = jl.put_journey(dict(id='llms', title='LLMs', goal='LLMs', steps=[dict(slug='tokenizers', status='todo'), dict(slug='embeddings', status='todo')]))
js = Session(Tutor(jl, fake2, context=False), journey='llms')
q = js.next()
test_eq((q.concept, q.why.split(';')[0]), ('tokenizers', 'the next step of your journey "LLMs"')) # a method note can follow
js.give_up(); jl.update('tokenizers', floor=4)
test_eq(js.next().concept, 'embeddings')
js.give_up(); jl.update('embeddings', floor=4)
test_eq((js.next(), jl.session(js.id).journey), (None, 'llms'))
```
The profile goes into the question prompt. At the `gentle` pace a new
concept starts at depth 1, not 3.
``` python
gl2 = temp_learning(); gl2.put('Ranking', 'Order items by a score.', 'ml')
gl2.set_profile(background='codes in Python', pace='gentle')
fk = Fake(dflt=dict(question='Which keeps its scale?', options=[dict(text='a rank', misconception=''), dict(text='a score', misconception='x')], correct=0, answer='', rubric=[]))
q = Tutor(gl2, fk, context=False).next()
test_eq(q.depth, 1)
assert 'codes in Python' in fk.calls[-1][0] and 'who' not in q
```
[`flow`](https://vedicreader.github.io/abhyasa/tutor.html#flow) moves
the pace by one step, and only from the last outcomes. A note comes only
when the pace changes.
``` python
sol, gap = dict(outcome='solid'), dict(outcome='gap')
test_eq(flow([]), ('steady', ''))
test_eq(flow([sol, gap, dict(outcome='misconception')]), ('gentle', 'easier: you missed the last two'))
test_eq(flow([sol] * 3, 'gentle'), ('steady', 'harder: 3 clean passes in a row'))
test_eq(flow([gap, gap], 'gentle'), ('gentle', '')) # already gentle: no change, no note
test_eq(flow([sol, sol, gap]), ('steady', '')) # one miss is not a trend
```
In a session, two misses make the next question easier, and its “why”
line says so.
[`Tutor.next`](https://vedicreader.github.io/abhyasa/tutor.html#tutor.next)
takes the pace from the session over the pace in the profile.
``` python
fw = temp_learning(); fw.put('Ranking', 'Order items by a score.', 'ml')
fs = Session(Tutor(fw, fake2, context=False), limit=5)
for _ in range(2): fs.next(); fs.give_up()
test_eq(fs.next().why.endswith('; easier: you missed the last two'), True)
fw.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
test_eq(Tutor(fw, fake2, context=False).next(subject='sanskrit', pace='gentle').depth, 1) # a new concept, at depth 1 and not 3
```
[`retests`](https://vedicreader.github.io/abhyasa/tutor.html#retests)
keeps a miss until a pass comes at least two questions later. A new miss
moves the concept to the end.
``` python
def lg(*xs): return [dict(concept=k, outcome=o, correct=o not in MISSES) for k, o in xs]
test_eq(retests(lg(('a', 'gap'), ('a', 'solid'))), {'a': 0}) # a pass right after: still to retest
test_eq(retests(lg(('a', 'gap'), ('b', 'solid'), ('b', 'solid'), ('a', 'fragile'))), {}) # a later pass clears it
test_eq(retests(lg(('a', 'gap'), ('b', 'misconception'), ('a', 'gap'))), {'b': 1, 'a': 2})
```
When the questions left are only enough for the retests, the session
asks the missed concept again, and says why. With more questions left,
it does not.
``` python
rl = temp_learning(); rl.put('Ranking', 'Order items by a score.', 'ml', order=0); rl.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', order=1)
rs = Session(Tutor(rl, fake2, context=False), limit=4)
rs.log = lg(('ranking', 'gap'), ('sandhi', 'solid'), ('sandhi', 'solid'))
q = rs.next()
test_eq((q.concept, q.why), ('ranking', 'a retest: you missed this earlier in this session'))
rs.tutor.open.clear(); rs.limit = 10
assert not rs.next().why.startswith('a retest')
```
[`bridges`](https://vedicreader.github.io/abhyasa/tutor.html#bridges)
takes known concepts only, from other subjects first. A transfer
question gets them in its prompt; an explain question does not.
``` python
bl = temp_learning()
bl.put('Attention', 'Weights from query-key dot products.', 'ml', related=['softmax'])
for n, s, f in (('Softmax', 'ml', 3), ('Ranking', 'ml', 4), ('Sandhi', 'sanskrit', 3), ('Nakshatra', 'jyotisha', 1)): bl.put(n, f'{n}.', s); bl.update(n.lower(), floor=f)
test_eq(bridges(bl.concept('attention'), bl.concepts()), ['Sandhi', 'Softmax', 'Ranking'])
fb = Fake(dflt=dict(question='Where else?', answer='', rubric=[]))
tb = Tutor(bl, fb, context=False)
tb.ask_on(bl.concept('attention'), 5); assert 'Sandhi' in fb.calls[-1][0]
tb.ask_on(bl.concept('attention'), 3); assert 'Sandhi' not in fb.calls[-1][0]
```
After each `observe_every` closed questions, the profile learns in the
background.
``` python
class Now:
"A pool that runs a job at once, for the test."
def submit(self, f, *a): f(*a)
ol = temp_learning(); ol.put('Ranking', 'Order items by a score.', 'ml')
fo = Fake({**fake2.rules, 'You keep a short profile': dict(changes=[dict(field='confuses', add='ranks against scores', why='You missed it twice.')])}, dflt=fake2.dflt)
ot = Tutor(ol, fo, context=False, pool=Now(), observe_every=2)
for _ in range(2): ot.give_up(ot.next().id)
test_eq(ol.profile().confuses, ['ranks against scores'])
```
The sources of the learner ground the question, the grade and the
lesson. First, import
[`passage`](https://vedicreader.github.io/abhyasa/ground.html#passage),
which makes one passage of a source.
``` python
from abhyasa.ground import passage
```
A source of notes grounds the question. The learner sees the citations
only when the question closes.
``` python
class Notes:
name = 'notes'
def search(self, q, k=4): return L([passage('vault:notes', 'n1', 'My ranking notes', 'Ranking orders items; RRF adds 1/(k + rank).')])
gl = temp_learning(); gl.put('Ranking', 'Order items by a score.', 'ml')
gt = Tutor(gl, fake2, sources=[Notes()])
q = gt.next()
assert 'My ranking notes' in fake2.calls[-1][0] # the question prompt has the passage
assert 'grounds' not in q and 'ctx' not in q # the learner does not see it yet
r = gt.answer(q.id, 'the rank position', confidence=80)
test_eq(r.grounds, [dict(src='vault:notes', ref='n1', title='My ranking notes', url='')])
assert 'My ranking notes' in fake2.calls[-1][0] # the grade prompt has it too
```
A function in `CLOSE_HOOKS` sees each closed question and can add to the
result. The result names its concept.
``` python
seen = []
def spy(t, q, r): seen.append(q.concept); r.extra = 1
CLOSE_HOOKS.append(spy)
try: r = gt.answer(gt.next().id, 'the rank position', confidence=80)
finally: CLOSE_HOOKS.remove(spy)
test_eq((seen, r.extra, r.concept), (['ranking'], 1, 'ranking'))
```
A follow-up is only what the learner says this time. The grade prompt
has the earlier turns in order, then the follow-up. A pass on the third
message, after two nudges, is `fragile`.
``` python
tl = temp_learning(); tl.put('Ranking', 'Order items by a score.', 'ml')
tt = Tutor(tl, fake, context=False)
q = tt.next()
tt.answer(q.id, 'sum the scores', confidence=60)
tt.answer(q.id, 'scale them first', confidence=60)
r = tt.answer(q.id, 'use the rank position', confidence=70)
p = next(c[0] for c in reversed(fake.calls) if 'You grade' in c[1])
assert p.index('Tutor (question)') < p.index('Learner: sum the scores') < p.index('Tutor (nudge)') < p.index('Learner: scale them first')
test_eq(p.split("Learner's answer: ")[1].split('\n')[0], 'use the rank position') # only the follow-up
assert 'scale them first' not in p.split("Learner's answer")[1]
test_eq((r.done, r.correct, r.outcome), (True, True, 'fragile'))
```
## Teach on request
After a shaky pass, a lesson comes only when you ask. It leaves the
learner state as the close left it, and a second ask gives the same
lesson.
``` python
sl = temp_learning(); sl.put('Ranking', 'Order items by a score.', 'ml')
ss = Session(Tutor(sl, fake, context=False), limit=3)
q = ss.next()
for a in ('sum the scores', 'scale them first', 'use the rank position'): r = ss.answer(a, confidence=70)
test_eq((r.done, r.outcome), (True, 'fragile'))
before = dict(sl.concept('ranking'))
```
The lesson builds on the first error, and the concept does not change.
``` python
lesson = ss.teach(q.id)
assert 'Check:' in lesson and 'could not' not in lesson
test_eq({k: v for k, v in sl.concept('ranking').items()}, before)
test_eq(ss.teach(q.id), lesson)
test_eq(len([x for x in ss.tutor.shut[q.id].turns if x.get('kind') == 'lesson']), 1)
```
The session log keeps the lesson, so a restart shows it.
``` python
test_eq(sl.session(ss.id).log[-1]['turns'][-1], dict(who='tutor', kind='lesson', text=lesson, method=ss.tutor.shut[q.id].method))
```
The grading prompt tells the model to explain a term that you say you do
not know, and to read past speech-to-text noise.
``` python
assert "I don't know what the loop means" in GRADE_SP and 'speech to text' in GRADE_SP
```
## Ways to learn, and gaps you name
Each question gets a method. The prompt says it, and the outcome counts
for it when the question closes.
``` python
import random
wl = temp_learning(); wl.put('Ranking', 'Order items by a score.', 'ml')
seen = []
def wgrade(p, sp):
seen.append(p); said = p.split("Learner's answer:")[1].split("Learner's think-aloud:")[0]
if 'the loop' in said: return dict(correct=False, grade=2, first_error='You skip the sum.', misconception='', feedback='A loop adds each product.',
gaps=['weighted sum as a loop'], asks_for='code')
return dict(correct='rank position' in said, grade=8, first_error='', misconception='', feedback='Yes.', gaps=[], asks_for='')
wfake = Fake({'You grade one answer': wgrade, 'The idea: weighted sum as a loop': dict(name='Weighted sum as a loop', statement='A loop adds each weight times its input.'),
'Teach the smallest step': 'Positions are shared. Check: what if k = 0?',
'Write ONE question': lambda p, sp: seen.append(p) or dict(question='Why do ranks merge well?', answer='They are comparable.', rubric=['comparable'])})
wt = Tutor(wl, wfake, context=False); wt.rng = random.Random(1)
q = wt.next()
assert q.method in METHODS and f"The way to teach this time: {METHODS[q.method]}" in seen[-1]
```
“I don’t know what the loop means” keeps the idea as a gap of the
concept, and “show me with code” switches the method for the next nudge.
``` python
r = wt.answer(q.id, "I don't know what the loop means, show me with code")
test_eq(wl.concept('ranking').gaps, ['weighted-sum-as-a-loop'])
test_eq(wt.open[q.id].method, 'code')
test_eq(wl.attempts('ranking')[0].gaps, ['weighted sum as a loop'])
r = wt.answer(q.id, 'use the rank position')
assert METHODS['code'] in seen[-1]
```
The close counts the outcome for the method. The gap now comes first on
the concept.
``` python
test_eq(wl.stats().methods['ml']['code'], [.5, .5]) # a pass after a wrong try is shaky: half a credit
test_eq(wt.next(focus='ranking').concept, 'weighted-sum-as-a-loop')
```
“Another way” writes a new lesson in a method not used yet on that
question, and records the request on the attempt.
``` python
l1 = wt.teach(q.id); l2 = wt.teach(q.id, another=True)
ms = [x['method'] for x in wt.shut[q.id].turns if x.get('kind') == 'lesson']
test_eq((len(ms), ms[0] != ms[1]), (2, True))
test_eq(wl.attempts('ranking')[0].taught, 2)
test_eq(wl.stats().methods['ml']['code'], [.5, .5]) # a lesson on request changes no counts
```
A drill session takes the weak concept first, says why, and keeps the
drill after a restart.
``` python
dl = temp_learning(); dl.put('Ranking', 'Order items by a score.', 'ml'); dl.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
dl.update('sandhi', misses=2, misconceptions=['vowels never change'], seen=2)
ds = Session(Tutor(dl, fake, context=False), drill=True, limit=4)
q = ds.next()
test_eq((q.concept, q.why.split(';')[0]), ('sandhi', 'a weak spot: a misconception: vowels never change'))
test_eq(Session.load(ds.tutor, ds.id).drill, True)
```