tutor

The loop: pick, ask, let the learner struggle, grade, update, teach the smallest step.

This module runs the learning loop. pick chooses the concept and the depth, and a model writes the question. The learner answers and can ask for hints. The model grades the answer, and the learner model updates the concept. After a miss, the tutor teaches the smallest step. Each surface (the web app, the ramabana terminal agent, a notebook) uses Tutor and Session.

Import the tools that this module uses, and the other abhyasa modules that the tutor calls.

Tries and hints

A wrong answer does not end the question. The tutor names the first error, gives no fix, and the learner tries again. After MAX_TRIES wrong tries, or when the learner stops, the tutor teaches the smallest step. Then the state of the concept records the miss. A pass after a wrong try counts as fragile.

Each open question keeps a thread, q.turns: the question, then each message of the learner and each reply of the tutor, in order. A message of the learner is only what they say this time, often a follow-up to the last nudge. They do not send the full answer again. The grade, the hints and the lesson read the whole thread.

HINTS is the hint ladder, from the least to the most revealing. The model gets one rung for each hint request.

DEPTH_ASK tells the model what a question at each depth must make the learner do.

TUTOR_SP is the system prompt for questions, hints and lessons.

Prompts

Each prompt function makes the text for one model call. The prompt text is an instruction to the model.

_concept_block describes the concept and the learner. question_prompt asks for one question at one depth. who is the profile of the learner as text (see profile), so the question uses their world.

question_prompt

def question_prompt(
    c, depth, avoid:tuple=(), ctx:str='', who:str='', bridge:tuple=(), method:str=''
):

The prompt that writes one question on c at depth in method; bridge names known ideas that a transfer question can connect to.

A transfer question asks the learner to use an idea in a new place. The best new place is one that they already know. bridges names up to three concepts that the learner knows to depth 3 or more: from other subjects first, then linked ones, then the deepest. Thus a transfer question on attention can use a Sanskrit or jyotiṣa idea.

bridges

def bridges(
    c, concepts, n:int=3
)->list:

The names of up to n concepts known to depth 3 or more, for a transfer question on c: other subjects first, then linked ones.

thread_text writes the thread as plain lines for a prompt. GRADE_SP and grade_prompt grade the conversation against the hidden model answer and the rubric. The latest message comes last, with the confidence that the learner gave it.

grade_prompt

def grade_prompt(
    q, answer, thinking:str='', confidence:NoneType=None
):

The prompt that grades the conversation on question q, with answer as the latest message of the learner.

thread_text

def thread_text(
    q
)->str:

The conversation on q so far, one turn to a line, in order.

hint_prompt asks for one rung of the hint ladder. It reads the thread, so the hint fits what the learner said.

hint_prompt

def hint_prompt(
    q, level
):

The prompt for hint level (0-5) on question q.

teach_prompt asks for the lesson after a miss. The lesson ends with a check question.

teach_prompt

def teach_prompt(
    q, first_error
):

The prompt that teaches the smallest step after a miss.

The tutor

Tutor keeps the open questions and the check questions from lessons. _ready_q finds a question that came with the source of a concept. After each observe_every closed questions, observe (from profile) reads the recent answers in the background and updates the profile.

Tutor

def Tutor(
    learning:abhyasa.store.Learning, ask, goal:int=4, context:bool=True, sources:NoneType=None, pool:NoneType=None,
    observe_every:int=6
):

The learning loop over a Learning store with an ask(prompt, sp) model; pool runs background jobs.

Ask a question

ask_on uses the first source that it finds: the check from the last lesson, a ready question, or a new question from the model. It shuffles the options of a choice question, and starts the thread with the question. public removes the fields that the learner must not see.

public

def public(
    q
)->fastcore.basics.AttrDict:

A question as the learner can see it: no model answer, no rubric, no correct option, no misconception tags.

Tutor.ask_on

def ask_on(
    c, depth:int, why:str='', session:str='', mode:str=None
)->fastcore.basics.AttrDict:

Write a question on concept c at depth in a method from choose, keep it open, and return it without its answer fields. mode='drill' uses your best method.

next lets pick choose the concept, then asks a question on it. paced moves the depth for the pace in the profile of the learner.

Tutor.next

def next(
    focus:str=None, subject:str=None, session:str='', skip:tuple=(), near:tuple=(), pace:str=None, mode:str=None
)->fastcore.basics.AttrDict | None:

The next question, chosen by pick at pace (by default the pace in the profile), or None when there is nothing to learn or review.

paced

def paced(
    c, depth:int, pace:str='steady'
)->int:

The depth for the pace of the learner: at gentle, at most one above the floor; at fast, one more than pick gives.

flow keeps a session in its band, at about 85% success. It reads the last outcomes of the session. After two misses in a row, the next question is one pace easier. After three solid passes in a row, it is one pace harder. The note says why, so the learner sees the change.

flow

def flow(
    log, pace:str='steady', n:int=3
)->tuple:

The (pace, note) for the next question of a session: one pace easier after two misses, one pace harder after n solid passes.

Hints

Each call to hint gives the next rung of the ladder. After the last rung, it gives the last hint again.

Tutor.hint

def hint(
    qid:str
)->str:

The next hint on the ladder, from the least to the most revealing.

Grade an answer

_grade grades a choice question in code and other questions with the model.

answer adds the message of the learner to the thread, and the nudge after a wrong try. A pass or the last try closes the question.

Tutor.answer

def answer(
    qid:str, answer:str='', thinking:str='', confidence:int=None, dont_know:bool=False
)->fastcore.basics.AttrDict:

Grade an answer. A wrong try keeps the question open with a nudge. A pass or the last try closes it and updates the concept.

Stop and teach

give_up closes the question as a miss. _close updates the concept, teaches after a miss and records the attempt. Then each function in CLOSE_HOOKS can add to the result. The demo module uses this hook to attach a demo after a miss. The confidence of the first try counts, because the first try shows what the learner believed.

Tutor.give_up

def give_up(
    qid:str, confidence:int=0, dont_know:bool=False
)->fastcore.basics.AttrDict:

Close the question as a miss and teach the smallest step. A shown answer always counts as a miss.

_lesson writes the lesson after a miss. When the model fails, the lesson is a note with the model answer.

The tutor keeps the last 20 closed questions in shut. teach gives a lesson on one of them when you ask for it, for example after a shaky pass. It does not change the learner state. A second call gives the same lesson.

Tutor.teach

def teach(
    qid:str, another:bool=False
)->str:

A lesson on request for the closed question qid, kept in its thread and on its attempt. another uses a different method. The learner state does not change.

A session

Session keeps one sitting together. Its id goes on each attempt, and it stops after limit questions (by default, the session length in the profile of the learner). It asks about one concept for at most run questions in a row. This lets the range between floor and ceiling close, and it keeps the learner interested.

A session is saved on the shelf after each move: when a question opens, after each answer and hint, and at the end. It keeps the log of each closed question (with the floor before and after, and the thread) and the open question with its thread. Thus Session.load continues it after a restart, with the same question, graded the same way.

A session in a journey asks about the current step of the journey. When the step is learned, the next question is on the next step.

Session

def Session(
    tutor:__main__.Tutor, focus:str=None, subject:str=None, limit:int=None, run:int=3, journey:str=None,
    sid:str=None, drill:bool=False
):

One sitting: questions and answers, saved on the shelf after each move, so that a restart can continue it.

A check right after a lesson shows that you understood it. It does not show that you will remember it, because the lesson is still in your mind. Thus each concept that you miss in a session comes back one more time before the end. At least two other questions come between the miss and the retest. retests finds these concepts. A pass right after the miss does not count.

retests

def retests(
    log, gap:int=2
)->dict:

The concepts missed in a session log, with no pass at least gap questions after the miss: concept -> its last miss, oldest first.

_retest asks one of them when only enough questions are left to retest each miss once.

A drill session (“Practice my weak spots”) asks your weakest concepts first, each one time, with your best method. When none is left, the session goes on as usual.

answer, give_up and hint act on the open question and save the session. A closed question goes in the log with what moved.

Session.hint

def hint()->str:

Session.give_up

def give_up(
    **kw
)->fastcore.basics.AttrDict:

Session.answer

def answer(
    *args, **kw
)->fastcore.basics.AttrDict:

save writes the session to the shelf. summary gives one line. end closes the session. load reads a session back, with its open question.

Session.load

def load(
    tutor:__main__.Tutor, sid:str
):

The session sid from the shelf, with its open question open again; None for an unknown id or an old summary.

Session.end

def end()->str:

Close the session and file it. Return its summary.

Session.save

def save():

File the session on the shelf, with its log and its open question. A session with nothing in it is not filed.

Session.summary

def summary()->str:

A short summary, and the text of the session document.

Session.teach asks for that lesson on a question of the session, and keeps it in the log, so the thread shows it after a restart.

Session.teach

def teach(
    qid:str, another:bool=False
)->str:

A lesson on request for qid, a closed question of this session; the log keeps it.

moved says what a session changed: for each concept, its floor before the first question and after the last one.

moved

def moved(
    log
)->list:

For each concept in a session log, in order: its name, the floor before and after, and the misconceptions found.

Tests

A fake model plays the tutor. Its grader passes an answer only when the answer has the words “rank position”. Two concepts are on the shelf, and Reciprocal rank fusion needs Ranking.

from abhyasa.store import temp_learning
from abhyasa.llm import Fake

lr = temp_learning()
lr.put('Ranking', 'Order items by a score.', 'ml', order=0)
lr.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank).', 'ml', prereqs=['ranking'], order=1)
def grader(p, sp): return dict(correct='rank position' in p.split("Learner's answer:")[1], grade=7, first_error='You add raw scores.',
                               misconception='scores are comparable across systems', feedback='Look at what each list really shares.')
fake = Fake({'You grade one answer': grader, 'Give hint level': 'Look at what the two lists have in common.',
             'Teach the smallest step': 'Only positions are shared. Check: what happens with k = 0?',
             'Write ONE question': dict(question='Why does RRF use ranks, not scores?', answer='Scores are not comparable.', rubric=['ranks are comparable'])})
t = Tutor(lr, fake, context=False)

/opt/hostedtoolcache/Python/3.12.15/x64/lib/python3.12/site-packages/usearch/__init__.py:131: UserWarning: Will download `usearch_sqlite` binary from GitHub.
  warnings.warn("Will download `usearch_sqlite` binary from GitHub.", UserWarning)

The first question is on the prereq, at depth 3. The learner does not see the answer or the rubric.

q = t.next()
test_eq((q.concept, q.depth, q.kind), ('ranking', 3, 'explain'))   # rrf waits for its prereq
assert 'answer' not in q and 'rubric' not in q

A wrong answer keeps the question open and names the first error. Then a hint comes from the ladder.

r = t.answer(q.id, 'sum the scores', confidence=90)
test_eq((r.done, r.correct, r.tries_left), (False, False, 2))
test_eq(r.first_error, 'You add raw scores.')
test_eq(t.hint(q.id), 'Look at what the two lists have in common.')

A pass after a wrong try and a hint is fragile. The floor stays, and the concept is shaky at depth 3.

r = t.answer(q.id, 'use the rank position', thinking='positions are shared', confidence=90)
test_eq((r.done, r.correct, r.outcome), (True, True, 'fragile'))   # passed, but after a wrong try and a hint
c = lr.concept('ranking')
test_eq((c.floor, c.shaky, c.seen), (0, 3, 1))
test_eq(len(lr.attempts('ranking')), 2)

The thread holds each turn in order: the question, the wrong try, its nudge, the hint and the pass.

test_eq([(x['who'], x.get('kind')) for x in q.turns],
        [('tutor', 'question'), ('you', None), ('tutor', 'nudge'), ('tutor', 'hint'), ('you', None), ('tutor', 'close')])
test_eq(lr.attempts('ranking')[0].thread[1]['text'], 'sum the scores')   # the final attempt keeps the thread

Three wrong tries close the question, keep the misconception and teach.

q = t.next(focus='ranking')
test_eq(q.depth, 3)    # shaky at explain: ask at explain again
for _ in range(3): r = t.answer(q.id, 'sum the scores', confidence=85)
test_eq((r.done, r.outcome), (True, 'misconception'))
assert 'Check:' in r.lesson
c = lr.concept('ranking')
test_eq((c.ceiling, c.misconceptions), (3, ['scores are comparable across systems']))

When the lesson model fails, the question still closes, the miss counts once, and the lesson says what happened.

fl = temp_learning(); fl.put('Ranking', 'Order items by a score.', 'ml')
def broken(p, sp=''):
    if 'Teach the smallest step' in p: raise RuntimeError('the model is down')
    return fake(p, sp)
ft = Tutor(fl, broken, context=False)
q = ft.next()
for a in ('sum the scores', 'add the scores', 'average the scores'): r = ft.answer(q.id, a, confidence=85)
test_eq((r.done, q.id in ft.open), (True, False))
assert 'could not write the lesson' in r.lesson and 'Scores are not comparable' in r.lesson
c = fl.concept('ranking')
test_eq((c.seen, c.misses), (1, 1))
test_eq(len([a for a in fl.attempts('ranking') if a.get('final')]), 1)

The check question from the lesson comes next.

q = t.next(focus='ranking')
test_eq(q.question, 'what happens with k = 0?')   # the lesson's check comes next
t.open.clear()

A ready question from a source comes before a question from the model.

lr.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', questions=[dict(q='Join na + asti.', a='nāsti', depth=3)])
q = t.next(subject='sanskrit')
test_eq(q.question, 'Join na + asti.')

Code grades a choice question. A wrong option gives the misconception that it matches.

fake.rules['Write ONE question'] = dict(question='Which is a sandhi?', options=[dict(text='na + asti → nāsti', misconception=''),
    dict(text='na + asti → naasti', misconception='vowels just sit side by side')], correct=0, answer='', rubric=[])
t.open.clear()
lr.update('sandhi', floor=0, ceiling=2)
q = t.next(subject='sanskrit')
test_eq(q.kind, 'choice')
r = t.answer(q.id, q.options.index('na + asti → naasti'), confidence=90)
test_eq(r.first_error, 'This option assumes: vowels just sit side by side')

A session stops after limit questions and files a summary.

fake2 = Fake({k: fake.rules[k] for k in ('You grade one answer', 'Teach the smallest step')},
             dflt=dict(question='What does a rank tell you?', answer='Its position.', rubric=['position']))
s = Session(Tutor(temp_learning(), fake2, context=False), limit=2)
s.tutor.learning.put('Ranking', 'Order items by a score.', 'ml')
assert s.next() is not None
s.give_up()
assert s.next() is not None
test_eq(s.answer('the rank position', confidence=80).correct, True)
test_eq(s.next(), None)
test_eq(s.summary(), '1 of 2 passed. Concepts: ranking.')
test_eq(s.tutor.learning.session(s.id).asked, 2)          # filed as it goes, not only at the end

A session on one concept stays in its subject. It asks about the neighbors of the concept first.

sl = temp_learning()
sl.put('Attention', 'Weights from query-key dot products.', 'ml', order=0)
sl.put('Softmax', 'Turns scores into weights that sum to 1.', 'ml', related=['attention'], order=5)
sl.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', order=0)
ss = Session(Tutor(sl, fake2, context=False), focus='attention')
test_eq((ss.subject, ss.near), ('ml', {'softmax'}))
sl.update('attention', floor=4, seen=1, due=1e12)   # learned, and not due: the session moves on, and stays in ml
test_eq(ss.next().concept, 'softmax')

A session is saved with its open question. load continues it in a new tutor, as after a restart, and grades the same question.

pl = temp_learning(); pl.put('Ranking', 'Order items by a score.', 'ml')
ps = Session(Tutor(pl, fake2, context=False), limit=3)
q1 = ps.next(); ps.give_up()
q2 = ps.next(); ps.answer('a score', confidence=60)       # a wrong try: the question stays open
t2 = Tutor(pl, fake2, context=False)                      # a restart: a new tutor with nothing open
back = Session.load(t2, ps.id)
test_eq((back.q.id, back.q.question, len(back.log), back.limit), (q2.id, q2.question, 1, 3))
assert 'answer' not in back.q and back.q.id in t2.open    # the learner view, and the full question in the tutor
test_eq(back.answer('the rank position', confidence=80).correct, True)
test_eq([(r['outcome'], r['floor0']) for r in back.log], [('gap', 0), ('fragile', 0)])
test_eq([t['who'] for t in back.log[-1]['turns']], ['tutor', 'you', 'tutor', 'you', 'tutor'])

moved gives the floor before and after for each concept of the log.

test_eq([(m['concept'], m['before'], m['after'], m['asked'], m['passed']) for m in moved(back.log)], [('ranking', 0, 0, 2, 1)])
test_eq(moved([dict(concept='a', name='A', floor0=2, floor=3, correct=True), dict(concept='a', floor0=3, floor=4, correct=True, misconception='x')]),
        [dict(concept='a', name='A', before=2, after=4, asked=2, passed=2, misconceptions=['x'])])

end closes the session. An ended session loads with no open question and gives no more questions.

back.next(); back.end()
again = Session.load(Tutor(pl, fake2, context=False), ps.id)
test_eq((again.q, again.next(), bool(again.ended)), (None, None, True))
test_is(Session.load(t2, 'nope'), None)

A session in a journey asks about the current step. When the step is learned, it goes to the next step. It stops at the end of the journey.

jl = temp_learning()
for n in ('Tokenizers', 'Embeddings', 'Sandhi'): jl.put(n, f'{n} statement.', 'ml' if n != 'Sandhi' else 'sanskrit')
jl.update('sandhi', seen=1, due=1.)                     # a review due elsewhere does not break in
j = jl.put_journey(dict(id='llms', title='LLMs', goal='LLMs', steps=[dict(slug='tokenizers', status='todo'), dict(slug='embeddings', status='todo')]))
js = Session(Tutor(jl, fake2, context=False), journey='llms')
q = js.next()
test_eq((q.concept, q.why.split(';')[0]), ('tokenizers', 'the next step of your journey "LLMs"'))   # a method note can follow
js.give_up(); jl.update('tokenizers', floor=4)
test_eq(js.next().concept, 'embeddings')
js.give_up(); jl.update('embeddings', floor=4)
test_eq((js.next(), jl.session(js.id).journey), (None, 'llms'))

The profile goes into the question prompt. At the gentle pace a new concept starts at depth 1, not 3.

gl2 = temp_learning(); gl2.put('Ranking', 'Order items by a score.', 'ml')
gl2.set_profile(background='codes in Python', pace='gentle')
fk = Fake(dflt=dict(question='Which keeps its scale?', options=[dict(text='a rank', misconception=''), dict(text='a score', misconception='x')], correct=0, answer='', rubric=[]))
q = Tutor(gl2, fk, context=False).next()
test_eq(q.depth, 1)
assert 'codes in Python' in fk.calls[-1][0] and 'who' not in q

flow moves the pace by one step, and only from the last outcomes. A note comes only when the pace changes.

sol, gap = dict(outcome='solid'), dict(outcome='gap')
test_eq(flow([]), ('steady', ''))
test_eq(flow([sol, gap, dict(outcome='misconception')]), ('gentle', 'easier: you missed the last two'))
test_eq(flow([sol] * 3, 'gentle'), ('steady', 'harder: 3 clean passes in a row'))
test_eq(flow([gap, gap], 'gentle'), ('gentle', ''))     # already gentle: no change, no note
test_eq(flow([sol, sol, gap]), ('steady', ''))           # one miss is not a trend

In a session, two misses make the next question easier, and its “why” line says so. Tutor.next takes the pace from the session over the pace in the profile.

fw = temp_learning(); fw.put('Ranking', 'Order items by a score.', 'ml')
fs = Session(Tutor(fw, fake2, context=False), limit=5)
for _ in range(2): fs.next(); fs.give_up()
test_eq(fs.next().why.endswith('; easier: you missed the last two'), True)
fw.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
test_eq(Tutor(fw, fake2, context=False).next(subject='sanskrit', pace='gentle').depth, 1)   # a new concept, at depth 1 and not 3

retests keeps a miss until a pass comes at least two questions later. A new miss moves the concept to the end.

def lg(*xs): return [dict(concept=k, outcome=o, correct=o not in MISSES) for k, o in xs]
test_eq(retests(lg(('a', 'gap'), ('a', 'solid'))), {'a': 0})                          # a pass right after: still to retest
test_eq(retests(lg(('a', 'gap'), ('b', 'solid'), ('b', 'solid'), ('a', 'fragile'))), {})   # a later pass clears it
test_eq(retests(lg(('a', 'gap'), ('b', 'misconception'), ('a', 'gap'))), {'b': 1, 'a': 2})

When the questions left are only enough for the retests, the session asks the missed concept again, and says why. With more questions left, it does not.

rl = temp_learning(); rl.put('Ranking', 'Order items by a score.', 'ml', order=0); rl.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', order=1)
rs = Session(Tutor(rl, fake2, context=False), limit=4)
rs.log = lg(('ranking', 'gap'), ('sandhi', 'solid'), ('sandhi', 'solid'))
q = rs.next()
test_eq((q.concept, q.why), ('ranking', 'a retest: you missed this earlier in this session'))
rs.tutor.open.clear(); rs.limit = 10
assert not rs.next().why.startswith('a retest')

bridges takes known concepts only, from other subjects first. A transfer question gets them in its prompt; an explain question does not.

bl = temp_learning()
bl.put('Attention', 'Weights from query-key dot products.', 'ml', related=['softmax'])
for n, s, f in (('Softmax', 'ml', 3), ('Ranking', 'ml', 4), ('Sandhi', 'sanskrit', 3), ('Nakshatra', 'jyotisha', 1)): bl.put(n, f'{n}.', s); bl.update(n.lower(), floor=f)
test_eq(bridges(bl.concept('attention'), bl.concepts()), ['Sandhi', 'Softmax', 'Ranking'])
fb = Fake(dflt=dict(question='Where else?', answer='', rubric=[]))
tb = Tutor(bl, fb, context=False)
tb.ask_on(bl.concept('attention'), 5); assert 'Sandhi' in fb.calls[-1][0]
tb.ask_on(bl.concept('attention'), 3); assert 'Sandhi' not in fb.calls[-1][0]

After each observe_every closed questions, the profile learns in the background.

class Now:
    "A pool that runs a job at once, for the test."
    def submit(self, f, *a): f(*a)
ol = temp_learning(); ol.put('Ranking', 'Order items by a score.', 'ml')
fo = Fake({**fake2.rules, 'You keep a short profile': dict(changes=[dict(field='confuses', add='ranks against scores', why='You missed it twice.')])}, dflt=fake2.dflt)
ot = Tutor(ol, fo, context=False, pool=Now(), observe_every=2)
for _ in range(2): ot.give_up(ot.next().id)
test_eq(ol.profile().confuses, ['ranks against scores'])

The sources of the learner ground the question, the grade and the lesson. First, import passage, which makes one passage of a source.

from abhyasa.ground import passage

A source of notes grounds the question. The learner sees the citations only when the question closes.

class Notes:
    name = 'notes'
    def search(self, q, k=4): return L([passage('vault:notes', 'n1', 'My ranking notes', 'Ranking orders items; RRF adds 1/(k + rank).')])
gl = temp_learning(); gl.put('Ranking', 'Order items by a score.', 'ml')
gt = Tutor(gl, fake2, sources=[Notes()])
q = gt.next()
assert 'My ranking notes' in fake2.calls[-1][0]                 # the question prompt has the passage
assert 'grounds' not in q and 'ctx' not in q                  # the learner does not see it yet
r = gt.answer(q.id, 'the rank position', confidence=80)
test_eq(r.grounds, [dict(src='vault:notes', ref='n1', title='My ranking notes', url='')])
assert 'My ranking notes' in fake2.calls[-1][0]                 # the grade prompt has it too

A function in CLOSE_HOOKS sees each closed question and can add to the result. The result names its concept.

seen = []
def spy(t, q, r): seen.append(q.concept); r.extra = 1
CLOSE_HOOKS.append(spy)
try: r = gt.answer(gt.next().id, 'the rank position', confidence=80)
finally: CLOSE_HOOKS.remove(spy)
test_eq((seen, r.extra, r.concept), (['ranking'], 1, 'ranking'))

A follow-up is only what the learner says this time. The grade prompt has the earlier turns in order, then the follow-up. A pass on the third message, after two nudges, is fragile.

tl = temp_learning(); tl.put('Ranking', 'Order items by a score.', 'ml')
tt = Tutor(tl, fake, context=False)
q = tt.next()
tt.answer(q.id, 'sum the scores', confidence=60)
tt.answer(q.id, 'scale them first', confidence=60)
r = tt.answer(q.id, 'use the rank position', confidence=70)
p = next(c[0] for c in reversed(fake.calls) if 'You grade' in c[1])
assert p.index('Tutor (question)') < p.index('Learner: sum the scores') < p.index('Tutor (nudge)') < p.index('Learner: scale them first')
test_eq(p.split("Learner's answer: ")[1].split('\n')[0], 'use the rank position')   # only the follow-up
assert 'scale them first' not in p.split("Learner's answer")[1]
test_eq((r.done, r.correct, r.outcome), (True, True, 'fragile'))

Teach on request

After a shaky pass, a lesson comes only when you ask. It leaves the learner state as the close left it, and a second ask gives the same lesson.

sl = temp_learning(); sl.put('Ranking', 'Order items by a score.', 'ml')
ss = Session(Tutor(sl, fake, context=False), limit=3)
q = ss.next()
for a in ('sum the scores', 'scale them first', 'use the rank position'): r = ss.answer(a, confidence=70)
test_eq((r.done, r.outcome), (True, 'fragile'))
before = dict(sl.concept('ranking'))

The lesson builds on the first error, and the concept does not change.

lesson = ss.teach(q.id)
assert 'Check:' in lesson and 'could not' not in lesson
test_eq({k: v for k, v in sl.concept('ranking').items()}, before)
test_eq(ss.teach(q.id), lesson)
test_eq(len([x for x in ss.tutor.shut[q.id].turns if x.get('kind') == 'lesson']), 1)

The session log keeps the lesson, so a restart shows it.

test_eq(sl.session(ss.id).log[-1]['turns'][-1], dict(who='tutor', kind='lesson', text=lesson, method=ss.tutor.shut[q.id].method))

The grading prompt tells the model to explain a term that you say you do not know, and to read past speech-to-text noise.

assert "I don't know what the loop means" in GRADE_SP and 'speech to text' in GRADE_SP

Ways to learn, and gaps you name

Each question gets a method. The prompt says it, and the outcome counts for it when the question closes.

import random
wl = temp_learning(); wl.put('Ranking', 'Order items by a score.', 'ml')
seen = []
def wgrade(p, sp):
    seen.append(p); said = p.split("Learner's answer:")[1].split("Learner's think-aloud:")[0]
    if 'the loop' in said: return dict(correct=False, grade=2, first_error='You skip the sum.', misconception='', feedback='A loop adds each product.',
                                    gaps=['weighted sum as a loop'], asks_for='code')
    return dict(correct='rank position' in said, grade=8, first_error='', misconception='', feedback='Yes.', gaps=[], asks_for='')
wfake = Fake({'You grade one answer': wgrade, 'The idea: weighted sum as a loop': dict(name='Weighted sum as a loop', statement='A loop adds each weight times its input.'),
              'Teach the smallest step': 'Positions are shared. Check: what if k = 0?',
              'Write ONE question': lambda p, sp: seen.append(p) or dict(question='Why do ranks merge well?', answer='They are comparable.', rubric=['comparable'])})
wt = Tutor(wl, wfake, context=False); wt.rng = random.Random(1)
q = wt.next()
assert q.method in METHODS and f"The way to teach this time: {METHODS[q.method]}" in seen[-1]

“I don’t know what the loop means” keeps the idea as a gap of the concept, and “show me with code” switches the method for the next nudge.

r = wt.answer(q.id, "I don't know what the loop means, show me with code")
test_eq(wl.concept('ranking').gaps, ['weighted-sum-as-a-loop'])
test_eq(wt.open[q.id].method, 'code')
test_eq(wl.attempts('ranking')[0].gaps, ['weighted sum as a loop'])
r = wt.answer(q.id, 'use the rank position')
assert METHODS['code'] in seen[-1]

The close counts the outcome for the method. The gap now comes first on the concept.

test_eq(wl.stats().methods['ml']['code'], [.5, .5])   # a pass after a wrong try is shaky: half a credit
test_eq(wt.next(focus='ranking').concept, 'weighted-sum-as-a-loop')

“Another way” writes a new lesson in a method not used yet on that question, and records the request on the attempt.

l1 = wt.teach(q.id); l2 = wt.teach(q.id, another=True)
ms = [x['method'] for x in wt.shut[q.id].turns if x.get('kind') == 'lesson']
test_eq((len(ms), ms[0] != ms[1]), (2, True))
test_eq(wl.attempts('ranking')[0].taught, 2)
test_eq(wl.stats().methods['ml']['code'], [.5, .5])   # a lesson on request changes no counts

A drill session takes the weak concept first, says why, and keeps the drill after a restart.

dl = temp_learning(); dl.put('Ranking', 'Order items by a score.', 'ml'); dl.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
dl.update('sandhi', misses=2, misconceptions=['vowels never change'], seen=2)
ds = Session(Tutor(dl, fake, context=False), drill=True, limit=4)
q = ds.next()
test_eq((q.concept, q.why.split(';')[0]), ('sandhi', 'a weak spot: a misconception: vowels never change'))
test_eq(Session.load(ds.tutor, ds.id).drill, True)