store

The learning shelf: concepts, attempts and sessions as vault documents.

This module keeps all learning data in your vault, on one shelf named learning. Each concept, attempt and session is one vault document. Thus each tool that reads the vault can search it. The other modules read and write the shelf only through the Learning class in this module.

Import the tools that this module uses, and the Vault class of the vault.

The shelf

Search reads the text of a document. The meta JSON keeps the state. Vault.set_meta changes the state and does not embed the text again. The table gives the source URI of each kind of document.

kind source
concept learning://concept/<slug>
attempt learning://attempt/<id>
session learning://session/<id>

These constants name the shelf, give the start state of a concept, and list the fields that merge.

slugify makes the key of a concept from its name. It changes a name in an Indic script, such as Devanagari, to IAST first. It changes +, # and & to words, so C, C++ and C# get different slugs. A name with no letters that it can read gets a short hash of the name. Thus the same name always gives the same slug. _src makes the source URI of a document.

source

slugify

def slugify(
    s:str
)->str:

Make a lower-case ASCII slug from s: saṃdhi and संधि give samdhi, and C++ gives c-plus-plus.

_union merges two lists and keeps the order. concept_text makes the text that search reads.

source

concept_text

def concept_text(
    c
)->str:

The searchable text of a concept: its name, statement, aliases and subject.

Learning opens one shelf of a vault. It also accepts a path, or None for the default vault.

source

Learning

def Learning(
    vault:NoneType=None, shelf:str='learning'
):

Concepts, attempts and sessions on one shelf of your vault.

Read concepts

Each method returns concepts as AttrDicts with the learner state filled in.

source

Learning.subjects

def subjects()->list:

The subjects that have concepts, with counts.

source

Learning.concepts

def concepts(
    subject:str=None
)->fastcore.foundation.L:

All concepts, or the concepts of one subject.

source

Learning.concept_by_doc

def concept_by_doc(
    doc_id:str
)->fastcore.basics.AttrDict | None:

The concept that a search hit (its doc_id) belongs to, or None when the hit is not a concept.

source

Learning.concept

def concept(
    slug:str
)->fastcore.basics.AttrDict | None:

One concept by slug, with its learner state.

Write concepts

put adds a concept or adds to it. It keeps the learner state, merges the list fields and replaces the other fields.

source

Learning.put

def put(
    name:str, # the concept's name
    statement:str='', # a true, self-contained statement of the idea
    subject:str='', # e.g. 'sanskrit', 'jyotisha', 'ml'
    slug:str=None, # defaults to `slugify(name)`
    prereqs:tuple=(), # slugs this concept needs first
    related:tuple=(), # slugs linked to it, in any direction
    aliases:tuple=(), # other names
    provenance:tuple=(), # where it came from: dicts such as `{'src': 'vedicreader', 'ref': ...}`
    questions:tuple=(), # ready questions: `{'q':..., 'a':..., 'depth':...}`
    **meta
)->fastcore.basics.AttrDict:

Add a concept or add to one. Keep the learner state, merge the list fields and replace the other fields.

update and link change one concept. drop removes it.

source

Learning.drop

def drop(
    slug:str
):

Remove a concept. Its attempts stay as a record of what happened.

source

source

Learning.update

def update(
    slug:str, **kv
)->fastcore.basics.AttrDict:

Change the state of a concept and do not embed its text again.

Move old slugs

abhyasa 0.0.5 and older made slugs in a different way. A name in Devanagari got a random slug, and C, C++ and C# got the same slug, c. reslug finds each concept that has an old slug. It moves the concept to the slug that slugify gives now. It also changes the slug in each place that points to it: the links of other concepts, attempts, sessions, journeys and demos. A slug that a source sets, such as sa-term-karaka, does not change.

reslug finds the moves first. With dry_run it stops there and writes nothing.

source

Learning.reslug

def reslug(
    dry_run:bool=False
)->dict:

Move each concept with an old-style slug to the slug from slugify, and every slug that points to it. Returns {old: new}.

Attempts

Each answer is one attempt document. Thus search can find what the learner said. The tutor can also find old questions and not ask them again.

source

attempt_text

def attempt_text(
    a
)->str:

The searchable text of an attempt: the question, the answer, the think-aloud and the feedback.

record saves one attempt.

source

Learning.record

def record(
    concept:str, question:str, answer:str='', thinking:str='', correct:bool=False, confidence:int=None, depth:int=1,
    kind:str='explain', hints:int=0, dont_know:bool=False, grade:int=None, feedback:str='', first_error:str='',
    misconception:str='', session:str='', **meta
)->fastcore.basics.AttrDict:

File one attempt as a vault document, so the learner can search what they said.

attempts reads them back, with the newest first.

source

Learning.attempts

def attempts(
    concept:str=None, session:str=None, limit:int=50
)->fastcore.foundation.L:

Recent attempts, newest first, for one concept or one session.

mark_attempt adds fields to a saved attempt, for example a lesson that you asked for later. add_gap keeps a gap that you named on a concept.

source

Learning.add_gap

def add_gap(
    slug:str, gap:str
):

Record that you said you do not know concept gap while you learned slug.

source

Learning.mark_attempt

def mark_attempt(
    aid:str, **kv
)->dict:

Merge kv into the meta of attempt aid.

Sessions

A session summary is one document. A new summary with the same id replaces the old one.

source

Learning.sessions

def sessions(
    limit:int=100
)->fastcore.foundation.L:

The filed sessions, the most recently changed first.

source

Learning.session

def session(
    sid:str
)->fastcore.basics.AttrDict | None:

One filed session by id, as its meta.

source

Learning.log_session

def log_session(
    sid:str, summary:str, **meta
):

File a session, and replace an earlier one with the same id. The meta changes also when the summary text does not.

Search and graph

search uses the hybrid search of the vault. graph gives the nodes and links for the graph view.

source

Learning.graph

def graph(
    subject:str=None
)->dict:

Nodes and links (prereq, related, gap) for a graph view. A link to a slug that has no concept is dropped.

source

Learning.search

def search(
    q:str, kind:str=None, limit:int=10
)->fastcore.foundation.L:

Hybrid search over the shelf: concepts, attempts and sessions.

A shelf for tests

Each test uses a vault in a temporary folder with the offline hashing encoder. The tests never open the real vault of the learner.

source

temp_learning

def temp_learning(
    **kw
)->__main__.Learning:

A Learning on a new vault in a temporary folder, with the offline hashing encoder.

Tests

put makes a concept with the start state and the slug from its name.

lr = temp_learning()
c = lr.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank) for each item.', 'ml', prereqs=['ranking'], aliases=['RRF'])
test_eq(c.slug, 'reciprocal-rank-fusion')
test_eq(c.floor, 0)
test_eq(c.prereqs, ['ranking'])

update changes the state.

c = lr.update(c.slug, floor=3, stability=2.)
test_eq((c.floor, c.stability), (3, 2.))

A second put keeps the learner state and merges the lists, also when the text changes.

c = lr.put('Reciprocal rank fusion', 'Merge ranked lists: score is the sum of 1/(k + rank).', related=['bm25'], aliases=['RRF', 'rank fusion'])
test_eq(c.floor, 3)
test_eq(c.subject, 'ml')
test_eq(c.aliases, ['RRF', 'rank fusion'])
test_eq(c.related, ['bm25'])
test_eq(len(lr.concepts()), 1)

concepts and subjects filter by subject. graph keeps only the links between known concepts.

lr.put('Ranking', 'Order items by a score.', 'ml')
lr.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
test_eq(len(lr.concepts('ml')), 2)
test_eq({r['subject'] for r in lr.subjects()}, {'ml', 'sanskrit'})
g = lr.graph()
test_eq(g['links'], [dict(source='ranking', target='reciprocal-rank-fusion', kind='prereq')])

record saves an attempt. attempts and search find it again.

a = lr.record('reciprocal-rank-fusion', 'Why does RRF ignore raw scores?', answer='Because scores from different systems are not comparable.',
              thinking='bm25 and cosine have different scales', correct=True, confidence=80, depth=3)
test_eq(lr.attempts('reciprocal-rank-fusion')[0].id, a.id)
test_eq(lr.attempts('ranking'), [])
test(lr.search('scales comparable', kind='attempt'), [], lambda a,b: len(a) > 0)

drop removes a concept.

lr.drop('ranking')
test_is(lr.concept('ranking'), None)

log_session saves a session. A second call with the same id replaces it. session reads it back.

lr.log_session('s1', '1 of 2 passed.', start=1., updated=2., asked=2)
lr.log_session('s1', '1 of 2 passed.', start=1., updated=3., asked=2, ended=3.)   # the same text: the meta still changes
test_eq((lr.session('s1').updated, lr.session('s1').ended, lr.session('s1').summary), (3., 3., '1 of 2 passed.'))
test_eq(len(lr.sessions()), 1)

sessions lists the most recently changed session first.

lr.log_session('s2', 'Started.', start=5., updated=5.)
test_eq(lr.sessions().attrgot('id'), ['s2', 's1'])
test_is(lr.session('nope'), None)

concept_by_doc gives the concept of a search hit. It gives None for a hit that is not a concept.

cl = temp_learning()
cl.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank).', 'ml')
cl.record('reciprocal-rank-fusion', 'Why ranks?', answer='scales differ')
test_eq(cl.concept_by_doc(cl.search('rank fusion', kind='concept')[0]['doc_id']).slug, 'reciprocal-rank-fusion')
test_is(cl.concept_by_doc(cl.search('scales differ', kind='attempt')[0]['doc_id']), None)

slugify reads Devanagari and Telugu, keeps C, C++ and C# apart, and gives the same hash slug for the same unreadable name.

test_eq([slugify(s) for s in ('saṃdhi', 'संधि', 'గుణ', 'C', 'C++', 'C#', 'R&D')], ['samdhi', 'samdhi', 'guna', 'c', 'c-plus-plus', 'c-sharp', 'r-and-d'])
test_eq(slugify('学习'), slugify('学习'))
assert slugify('学习').startswith('c-')

This test makes a vault for reslug as abhyasa 0.0.5 left it: C# saved as c, a Devanagari name with a random slug, and links, an attempt and a session that use these slugs.

ol = temp_learning()
ol.put('C#', 'A language from Microsoft.', 'code', slug='c')
ol.put('संधि', 'Sounds change where words meet.', 'sanskrit', slug='a1b2c3d4')
ol.put('karaka term', 'A set slug.', 'sanskrit', slug='sa-term-karaka')
ol.put('Ranking', 'Order items by a score.', 'ml', prereqs=['c'], related=['a1b2c3d4', 'sa-term-karaka'])
ol.record('c', 'What is C#?', answer='c')
ol.log_session('s1', 'One question.', concepts=['c'], log=[dict(concept='c', question='What is C#?')], focus='a1b2c3d4')

A dry run lists the moves and changes nothing. A slug that a source sets is not in the list.

test_eq(ol.reslug(dry_run=True), {'c': 'c-sharp', 'a1b2c3d4': 'samdhi'})
assert ol.concept('c') is not None and ol.concept('c-sharp') is None

The real run moves the concepts and every slug that points to them. An answer that is the text c stays as it is. A second run has nothing to do.

test_eq(ol.reslug(), {'c': 'c-sharp', 'a1b2c3d4': 'samdhi'})
test_eq((ol.concept('c'), ol.concept('a1b2c3d4')), (None, None))
test_eq((ol.concept('c-sharp').name, ol.concept('samdhi').statement), ('C#', 'Sounds change where words meet.'))
r = ol.concept('ranking')
test_eq((r.prereqs, r.related), (['c-sharp'], ['samdhi', 'sa-term-karaka']))
a = ol.attempts('c-sharp')[0]
test_eq((a.concept, a.answer), ('c-sharp', 'c'))
s = ol.session('s1')
test_eq((s.concepts, s.log[0]['concept'], s.focus), (['c-sharp'], 'c-sharp', 'samdhi'))
test_eq(ol.reslug(), {})

mark_attempt changes a saved attempt. add_gap keeps each gap one time, and graph shows it as a gap link.

gl = temp_learning()
gl.put('Functions and the neuron', 'z = w . x + b.', 'ml'); gl.put('Weighted sum', 'Add each input times its weight.', 'ml')
a = gl.record('functions-and-the-neuron', 'Why does z double?', answer='2z')
test_eq(gl.mark_attempt(a.id, taught=True)['taught'], True)
test_eq(gl.attempts('functions-and-the-neuron')[0].taught, True)
gl.add_gap('functions-and-the-neuron', 'weighted-sum'); gl.add_gap('functions-and-the-neuron', 'weighted-sum')
test_eq(gl.concept('functions-and-the-neuron').gaps, ['weighted-sum'])
assert dict(source='weighted-sum', target='functions-and-the-neuron', kind='gap') in gl.graph()['links']