model

The learner model: how deep you know each concept, when to review it, and what to ask next.

This module holds the learner model as plain functions with no model calls and no storage. It finds what an answer means, updates the state of a concept, finds the next depth and picks the next concept. The tutor calls these functions and the store keeps their results.

Import the tools that this module uses.

Depth

Each concept has a depth from 0 to 5. The tutor asks each question at one depth.

depth	name	a question at this depth asks you to
0	`unseen`	(no question)
1	`recognise`	pick the correct statement from options
2	`recall`	state the idea or the rule
3	`explain`	say why it is true, in your own words
4	`apply`	use it to solve a problem
5	`transfer`	use it in a new case or link it to another subject

The floor is the highest depth that you pass reliably. The ceiling is the lowest depth that you failed (6 until you fail). The next question goes between them. Thus each answer moves one edge of this range.

These constants name the depths, give the question kind at each depth, and set the goal and the high-confidence line.

Outcome

Correctness and confidence together tell what an answer means.

	high confidence	low confidence
correct	`solid`	`fragile`
wrong	`misconception`	`gap`

“I don’t know” is a gap. A correct answer that used two or more hints is fragile.

source

outcome

def outcome(
    correct:bool, confidence:int=None, hints:int=0, dont_know:bool=False
)->str:

One of solid, fragile, misconception or gap. With no confidence, only the answer and the hints count.

A correct answer is solid with high confidence. Two hints or low confidence make it fragile.

test_eq(outcome(True, 80), 'solid')
test_eq(outcome(True, 80, hints=2), 'fragile')
test_eq(outcome(True, 30), 'fragile')

With no confidence given, a clean pass is solid and a wrong answer is a gap, never a misconception.

test_eq([outcome(True), outcome(True, hints=2), outcome(False)], ['solid', 'fragile', 'gap'])

A wrong answer is a misconception with high confidence, else a gap. “I don’t know” is always a gap.

test_eq(outcome(False, 90), 'misconception')
test_eq(outcome(False, 20), 'gap')
test_eq(outcome(True, 90, dont_know=True), 'gap')

Memory

Recall falls with time as in FSRS: \(R(t) = (1 + \frac{19}{81}\frac{t}{S})^{-1/2}\). The time \(t\) and the stability \(S\) are in days. At \(t = S\), recall is 90%. Thus the due date is the last review plus \(S\) days.

recall gives the recall now. grow gives the new stability after an outcome.

source

grow

def grow(
    s:float, r:float, kind:str
)->float:

The new stability after an outcome, from the old stability s and the recall r at the time of the answer.

source

recall

def recall(
    c, now:float=None
)->float:

The chance, from 0 to 1, that you still recall concept c now.

Recall is 90% after \(S\) days, and 0 for a concept with no review.

test_close(recall(dict(stability=2., last=0.), now=2*DAY), .9, eps=1e-6)
test_eq(recall(dict(stability=0, last=None)), 0.)

A pass grows stability. A harder recall grows it more. A gap shrinks it.

test_eq(grow(0, 0, 'solid'), 1.)
assert grow(4, .7, 'solid') > grow(4, .99, 'solid') > 4   # a review that was harder to recall teaches more
assert grow(4, .9, 'gap') < 4

Update

update returns the fields to change after one answer at depth d.

outcome	change
`solid`	The floor rises to `d`. The streak grows.
`fragile`	The floor stays. The concept is `shaky` at `d`, so the next question is at `d` again.
`misconception`	The ceiling falls to `d`. The floor falls below `d`. The misconception is kept.
`gap`	The ceiling falls to `d`. The floor falls below `d`.

source

update

def update(
    c, depth:int, correct:bool, confidence:int=None, hints:int=0, dont_know:bool=False, misconception:str='',
    now:float=None
)->fastcore.basics.AttrDict:

The state changes for concept c after one answer at depth, with the outcome in kind.

A solid pass at depth 3 on a new concept raises the floor to 3. The review is due in one day.

c = AttrDict(floor=0, ceiling=6, stability=0., last=None, streak=0, misses=0, seen=0)
u = update(c, 3, True, 80, now=0.)
test_eq((u.kind, u.floor, u.ceiling, u.streak, u.due), ('solid', 3, 6, 1, DAY))
c.update(u)

A confident wrong answer at depth 4 lowers the ceiling to 4 and keeps the misconception.

u = update(c, 4, False, 85, misconception='thinks RRF needs normalised scores', now=DAY)
test_eq((u.kind, u.floor, u.ceiling, u.misses), ('misconception', 3, 4, 1))
test_eq(u.misconceptions, ['thinks RRF needs normalised scores'])
c.update(u)

A miss below the floor lowers the floor. A low-confidence pass makes the concept shaky at that depth.

u = update(c, 2, False, 10, now=DAY)    # a miss below the floor: the floor falls
test_eq((u.kind, u.floor, u.ceiling), ('gap', 1, 2))
test_eq(update(c, 4, True, 30).shaky, 4)

The next depth

Ask in the middle of the range between floor and ceiling, so one answer cuts the range in half. Ask a shaky concept at the same depth again. When the ceiling is the floor plus 1, ask at the ceiling, because the work is at that edge.

source

next_depth

def next_depth(
    c
)->int:

The depth for the next question on concept c.

A new concept starts at depth 3 (explain). An open range gives its middle.

test_eq(next_depth(dict(floor=0, ceiling=6)), 3)     # a new concept: start at explain, not at the bottom
test_eq(next_depth(dict(floor=3, ceiling=6)), 4)
test_eq(next_depth(dict(floor=0, ceiling=3)), 1)
test_eq(next_depth(dict(floor=1, ceiling=3)), 2)

A closed range gives the ceiling. Depth 5 is the top. A shaky depth comes first.

test_eq(next_depth(dict(floor=2, ceiling=3)), 3)
test_eq(next_depth(dict(floor=5, ceiling=6)), 5)
test_eq(next_depth(dict(floor=1, ceiling=6, shaky=4)), 4)

What to ask next

pick chooses the concept. Code makes the choice, not the model, so the app can show the reason.

A focus concept, when you chose one and did not learn it yet.
A review that is due, with the lowest recall first.
The frontier: concepts below the goal with all prereqs at depth 3 or more. Concepts that you started come first.

After two misses in a row on a concept, pick goes back to its weakest prereq that is below the goal.

learned and ready test one concept. _back finds the prereq to go back to.

source

ready

def ready(
    c, by:dict
)->bool:

Whether each prereq of c that is a known concept is at depth 3 or more.

source

learned

def learned(
    c, goal:int=4
)->bool:

pick returns the concept, the depth and the reason.

source

pick

def pick(
    concepts, focus:str=None, subject:str=None, goal:int=4, now:float=None, skip:tuple=(), near:tuple=()
):

The next (concept, depth, why), or None when no review is due and the frontier is empty. Concepts in near come first on the frontier.

Strengths and weak spots

understanding reads one concept and tells how well you know it: the floor, the ceiling, the recall now, and the reasons that it is weak. It uses only the state of the concept. Thus a page of all concepts does not need to read the attempts. weak is true when a reason says that you need help now. strong needs depth 3 or more, recall of 80% or more, and no reasons.

source

weak

def weak(
    c, now:float=None
)->bool:

source

understanding

def understanding(
    c, now:float=None
)->fastcore.basics.AttrDict:

How well you know concept c: floor, ceiling, recall, the reasons it is weak, weak, strong, one state, and a score to sort by.

A concept with a misconception and two misses is weak, with both reasons. A deep, fresh concept with no misses is strong.

now = 1e6
u = understanding(dict(slug='a', floor=1, ceiling=3, misses=2, misconceptions=['scores add'], seen=3, stability=5., last=now), now)
test_eq((u.weak, u.strong, u.reasons[:2]), (True, False, ['a misconception: scores add', 'missed the last 2 times']))
s = understanding(dict(slug='b', floor=4, ceiling=6, seen=5, stability=30., last=now - DAY), now)
test_eq((s.weak, s.strong, s.reasons), (False, True, []))
test_eq(understanding(dict(slug='g', floor=0, provenance=[dict(src='gap')]), now).reasons, ['you said you did not know it'])
assert understanding(dict(slug='f', floor=3, seen=4, stability=1., last=now - 30 * DAY), now).reasons[0].startswith('fading')
test_eq([u.state, s.state, understanding(dict(slug='n')).state], ['weak', 'strong', 'unseen'])

Three concepts: b needs a. The first pick is a, because b waits for its prereq.

def C(slug, **kw): return AttrDict(slug=slug, name=slug, **{**dict(floor=0, ceiling=6, seen=0, misses=0, prereqs=[]), **kw})
cs = [C('a', order=0), C('b', prereqs=['a'], order=1), C('c', order=2)]
test_eq(pick(cs)[0].slug, 'a')                          # b waits for a

A started concept below the goal comes first. When a is learned, b is ready.

cs[0].update(floor=3, seen=1, due=10*DAY, stability=10., last=0.)
test_eq(pick(cs, now=DAY)[0].slug, 'a')                 # a was started and is below the goal
cs[0].update(floor=4)
test_eq(pick(cs, now=DAY)[0].slug, 'b')                 # a is learned; b is ready

A due review comes before the frontier, at the floor of the concept.

r = pick(cs, now=11*DAY)
test_eq((r[0].slug, r[1]), ('a', 4))                    # a is due: a review at its floor
assert 'due for review' in r[2]

Two misses go back to the weakest prereq. A learned concept that is not due gives no question.

cs[1].update(misses=2, seen=2, floor=1, ceiling=2); cs[0].update(floor=3, due=None)
test_eq(pick(cs, focus='b', now=DAY)[0].slug, 'a')      # two misses on b: step back to a
test_eq(pick([C('x', floor=4, seen=1, due=1e12)]), None)

A concept in near comes first on the frontier, before a concept you started elsewhere.

cs = [C('far', order=0, seen=1), C('next', order=5)]
test_eq(pick(cs)[0].slug, 'far')
r = pick(cs, near={'next'})
test_eq(r[0].slug, 'next')
assert 'next to the concept you chose' in r[2]

A gap that you named on a concept comes first, at the easiest depth. After you pass it, or after two questions on it, the concept comes back.

nn = dict(slug='neuron', name='Neuron', floor=0, ceiling=6, gaps=['wsum'])
ws = dict(slug='wsum', name='Weighted sum', floor=0, ceiling=6, seen=0)
c, d, why = pick([nn, ws], focus='neuron')
test_eq((c['slug'], d), ('wsum', 1)); assert 'you said you did not know Weighted sum' in why
test_eq(pick([nn, dict(ws, floor=1, seen=1)], focus='neuron')[0]['slug'], 'neuron')
test_eq(pick([nn, dict(ws, seen=2)], focus='neuron')[0]['slug'], 'neuron')