# store
This module keeps all learning data in your vault, on one shelf named
`learning`. Each concept, attempt and session is one vault document.
Thus each tool that reads the vault can search it. The other modules
read and write the shelf only through the
[`Learning`](https://vedicreader.github.io/abhyasa/store.html#learning)
class in this module.
Import the tools that this module uses, and the `Vault` class of the
vault.
## The shelf
Search reads the text of a document. The `meta` JSON keeps the state.
`Vault.set_meta` changes the state and does not embed the text again.
The table gives the source URI of each kind of document.
kind
source
concept
learning://concept/<slug>
attempt
learning://attempt/<id>
session
learning://session/<id>
These constants name the shelf, give the start state of a concept, and
list the fields that merge.
[`slugify`](https://vedicreader.github.io/abhyasa/store.html#slugify)
makes the key of a concept from its name. It changes a name in an Indic
script, such as Devanagari, to IAST first. It changes `+`, `#` and `&`
to words, so C, C++ and C# get different slugs. A name with no letters
that it can read gets a short hash of the name. Thus the same name
always gives the same slug.
[`_src`](https://vedicreader.github.io/abhyasa/store.html#_src) makes
the source URI of a document.
------------------------------------------------------------------------
source
### slugify
``` python
def slugify(
s:str
)->str:
```
*Make a lower-case ASCII slug from `s`: `saṃdhi` and `संधि` give
`samdhi`, and `C++` gives `c-plus-plus`.*
[`_union`](https://vedicreader.github.io/abhyasa/store.html#_union)
merges two lists and keeps the order.
[`concept_text`](https://vedicreader.github.io/abhyasa/store.html#concept_text)
makes the text that search reads.
------------------------------------------------------------------------
source
### concept_text
``` python
def concept_text(
c
)->str:
```
*The searchable text of a concept: its name, statement, aliases and
subject.*
[`Learning`](https://vedicreader.github.io/abhyasa/store.html#learning)
opens one shelf of a vault. It also accepts a path, or `None` for the
default vault.
------------------------------------------------------------------------
source
### Learning
``` python
def Learning(
vault:NoneType=None, shelf:str='learning'
):
```
*Concepts, attempts and sessions on one shelf of your vault.*
## Read concepts
Each method returns concepts as `AttrDict`s with the learner state
filled in.
------------------------------------------------------------------------
source
### Learning.subjects
``` python
def subjects()->list:
```
*The subjects that have concepts, with counts.*
------------------------------------------------------------------------
source
### Learning.concepts
``` python
def concepts(
subject:str=None
)->fastcore.foundation.L:
```
*All concepts, or the concepts of one subject.*
------------------------------------------------------------------------
source
### Learning.concept_by_doc
``` python
def concept_by_doc(
doc_id:str
)->fastcore.basics.AttrDict | None:
```
*The concept that a search hit (its `doc_id`) belongs to, or None when
the hit is not a concept.*
------------------------------------------------------------------------
source
### Learning.concept
``` python
def concept(
slug:str
)->fastcore.basics.AttrDict | None:
```
*One concept by slug, with its learner state.*
## Write concepts
`put` adds a concept or adds to it. It keeps the learner state, merges
the list fields and replaces the other fields.
------------------------------------------------------------------------
source
### Learning.put
``` python
def put(
name:str, # the concept's name
statement:str='', # a true, self-contained statement of the idea
subject:str='', # e.g. 'sanskrit', 'jyotisha', 'ml'
slug:str=None, # defaults to `slugify(name)`
prereqs:tuple=(), # slugs this concept needs first
related:tuple=(), # slugs linked to it, in any direction
aliases:tuple=(), # other names
provenance:tuple=(), # where it came from: dicts such as `{'src': 'vedicreader', 'ref': ...}`
questions:tuple=(), # ready questions: `{'q':..., 'a':..., 'depth':...}`
**meta
)->fastcore.basics.AttrDict:
```
*Add a concept or add to one. Keep the learner state, merge the list
fields and replace the other fields.*
[`update`](https://vedicreader.github.io/abhyasa/model.html#update) and
`link` change one concept. `drop` removes it.
------------------------------------------------------------------------
source
### Learning.drop
``` python
def drop(
slug:str
):
```
*Remove a concept. Its attempts stay as a record of what happened.*
------------------------------------------------------------------------
source
### Learning.link
``` python
def link(
slug:str, prereq:str
):
```
*Record that `prereq` comes before `slug`.*
------------------------------------------------------------------------
source
### Learning.update
``` python
def update(
slug:str, **kv
)->fastcore.basics.AttrDict:
```
*Change the state of a concept and do not embed its text again.*
## Move old slugs
abhyasa 0.0.5 and older made slugs in a different way. A name in
Devanagari got a random slug, and C, C++ and C# got the same slug, `c`.
`reslug` finds each concept that has an old slug. It moves the concept
to the slug that
[`slugify`](https://vedicreader.github.io/abhyasa/store.html#slugify)
gives now. It also changes the slug in each place that points to it: the
links of other concepts, attempts, sessions, journeys and demos. A slug
that a source sets, such as `sa-term-karaka`, does not change.
`reslug` finds the moves first. With `dry_run` it stops there and writes
nothing.
------------------------------------------------------------------------
source
### Learning.reslug
``` python
def reslug(
dry_run:bool=False
)->dict:
```
*Move each concept with an old-style slug to the slug from
[`slugify`](https://vedicreader.github.io/abhyasa/store.html#slugify),
and every slug that points to it. Returns `{old: new}`.*
## Attempts
Each answer is one attempt document. Thus search can find what the
learner said. The tutor can also find old questions and not ask them
again.
------------------------------------------------------------------------
source
### attempt_text
``` python
def attempt_text(
a
)->str:
```
*The searchable text of an attempt: the question, the answer, the
think-aloud and the feedback.*
`record` saves one attempt.
------------------------------------------------------------------------
source
### Learning.record
``` python
def record(
concept:str, question:str, answer:str='', thinking:str='', correct:bool=False, confidence:int=None, depth:int=1,
kind:str='explain', hints:int=0, dont_know:bool=False, grade:int=None, feedback:str='', first_error:str='',
misconception:str='', session:str='', **meta
)->fastcore.basics.AttrDict:
```
*File one attempt as a vault document, so the learner can search what
they said.*
`attempts` reads them back, with the newest first.
------------------------------------------------------------------------
source
### Learning.attempts
``` python
def attempts(
concept:str=None, session:str=None, limit:int=50
)->fastcore.foundation.L:
```
*Recent attempts, newest first, for one concept or one session.*
`mark_attempt` adds fields to a saved attempt, for example a lesson that
you asked for later. `add_gap` keeps a gap that you named on a concept.
------------------------------------------------------------------------
source
### Learning.add_gap
``` python
def add_gap(
slug:str, gap:str
):
```
*Record that you said you do not know concept `gap` while you learned
`slug`.*
------------------------------------------------------------------------
source
### Learning.mark_attempt
``` python
def mark_attempt(
aid:str, **kv
)->dict:
```
*Merge `kv` into the meta of attempt `aid`.*
## Sessions
A session summary is one document. A new summary with the same id
replaces the old one.
------------------------------------------------------------------------
source
### Learning.sessions
``` python
def sessions(
limit:int=100
)->fastcore.foundation.L:
```
*The filed sessions, the most recently changed first.*
------------------------------------------------------------------------
source
### Learning.session
``` python
def session(
sid:str
)->fastcore.basics.AttrDict | None:
```
*One filed session by id, as its meta.*
------------------------------------------------------------------------
source
### Learning.log_session
``` python
def log_session(
sid:str, summary:str, **meta
):
```
*File a session, and replace an earlier one with the same id. The meta
changes also when the summary text does not.*
## Search and graph
`search` uses the hybrid search of the vault. `graph` gives the nodes
and links for the graph view.
------------------------------------------------------------------------
source
### Learning.graph
``` python
def graph(
subject:str=None
)->dict:
```
*Nodes and links (prereq, related, gap) for a graph view. A link to a
slug that has no concept is dropped.*
------------------------------------------------------------------------
source
### Learning.search
``` python
def search(
q:str, kind:str=None, limit:int=10
)->fastcore.foundation.L:
```
*Hybrid search over the shelf: concepts, attempts and sessions.*
## A shelf for tests
Each test uses a vault in a temporary folder with the offline hashing
encoder. The tests never open the real vault of the learner.
------------------------------------------------------------------------
source
### temp_learning
``` python
def temp_learning(
**kw
)->__main__.Learning:
```
*A
[`Learning`](https://vedicreader.github.io/abhyasa/store.html#learning)
on a new vault in a temporary folder, with the offline hashing encoder.*
## Tests
`put` makes a concept with the start state and the slug from its name.
``` python
lr = temp_learning()
c = lr.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank) for each item.', 'ml', prereqs=['ranking'], aliases=['RRF'])
test_eq(c.slug, 'reciprocal-rank-fusion')
test_eq(c.floor, 0)
test_eq(c.prereqs, ['ranking'])
```
[`update`](https://vedicreader.github.io/abhyasa/model.html#update)
changes the state.
``` python
c = lr.update(c.slug, floor=3, stability=2.)
test_eq((c.floor, c.stability), (3, 2.))
```
A second `put` keeps the learner state and merges the lists, also when
the text changes.
``` python
c = lr.put('Reciprocal rank fusion', 'Merge ranked lists: score is the sum of 1/(k + rank).', related=['bm25'], aliases=['RRF', 'rank fusion'])
test_eq(c.floor, 3)
test_eq(c.subject, 'ml')
test_eq(c.aliases, ['RRF', 'rank fusion'])
test_eq(c.related, ['bm25'])
test_eq(len(lr.concepts()), 1)
```
`concepts` and `subjects` filter by subject. `graph` keeps only the
links between known concepts.
``` python
lr.put('Ranking', 'Order items by a score.', 'ml')
lr.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
test_eq(len(lr.concepts('ml')), 2)
test_eq({r['subject'] for r in lr.subjects()}, {'ml', 'sanskrit'})
g = lr.graph()
test_eq(g['links'], [dict(source='ranking', target='reciprocal-rank-fusion', kind='prereq')])
```
`record` saves an attempt. `attempts` and `search` find it again.
``` python
a = lr.record('reciprocal-rank-fusion', 'Why does RRF ignore raw scores?', answer='Because scores from different systems are not comparable.',
thinking='bm25 and cosine have different scales', correct=True, confidence=80, depth=3)
test_eq(lr.attempts('reciprocal-rank-fusion')[0].id, a.id)
test_eq(lr.attempts('ranking'), [])
test(lr.search('scales comparable', kind='attempt'), [], lambda a,b: len(a) > 0)
```
`drop` removes a concept.
``` python
lr.drop('ranking')
test_is(lr.concept('ranking'), None)
```
`log_session` saves a session. A second call with the same id replaces
it. `session` reads it back.
``` python
lr.log_session('s1', '1 of 2 passed.', start=1., updated=2., asked=2)
lr.log_session('s1', '1 of 2 passed.', start=1., updated=3., asked=2, ended=3.) # the same text: the meta still changes
test_eq((lr.session('s1').updated, lr.session('s1').ended, lr.session('s1').summary), (3., 3., '1 of 2 passed.'))
test_eq(len(lr.sessions()), 1)
```
`sessions` lists the most recently changed session first.
``` python
lr.log_session('s2', 'Started.', start=5., updated=5.)
test_eq(lr.sessions().attrgot('id'), ['s2', 's1'])
test_is(lr.session('nope'), None)
```
`concept_by_doc` gives the concept of a search hit. It gives None for a
hit that is not a concept.
``` python
cl = temp_learning()
cl.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank).', 'ml')
cl.record('reciprocal-rank-fusion', 'Why ranks?', answer='scales differ')
test_eq(cl.concept_by_doc(cl.search('rank fusion', kind='concept')[0]['doc_id']).slug, 'reciprocal-rank-fusion')
test_is(cl.concept_by_doc(cl.search('scales differ', kind='attempt')[0]['doc_id']), None)
```
[`slugify`](https://vedicreader.github.io/abhyasa/store.html#slugify)
reads Devanagari and Telugu, keeps C, C++ and C# apart, and gives the
same hash slug for the same unreadable name.
``` python
test_eq([slugify(s) for s in ('saṃdhi', 'संधि', 'గుణ', 'C', 'C++', 'C#', 'R&D')], ['samdhi', 'samdhi', 'guna', 'c', 'c-plus-plus', 'c-sharp', 'r-and-d'])
test_eq(slugify('学习'), slugify('学习'))
assert slugify('学习').startswith('c-')
```
This test makes a vault for `reslug` as abhyasa 0.0.5 left it: C# saved
as `c`, a Devanagari name with a random slug, and links, an attempt and
a session that use these slugs.
``` python
ol = temp_learning()
ol.put('C#', 'A language from Microsoft.', 'code', slug='c')
ol.put('संधि', 'Sounds change where words meet.', 'sanskrit', slug='a1b2c3d4')
ol.put('karaka term', 'A set slug.', 'sanskrit', slug='sa-term-karaka')
ol.put('Ranking', 'Order items by a score.', 'ml', prereqs=['c'], related=['a1b2c3d4', 'sa-term-karaka'])
ol.record('c', 'What is C#?', answer='c')
ol.log_session('s1', 'One question.', concepts=['c'], log=[dict(concept='c', question='What is C#?')], focus='a1b2c3d4')
```
A dry run lists the moves and changes nothing. A slug that a source sets
is not in the list.
``` python
test_eq(ol.reslug(dry_run=True), {'c': 'c-sharp', 'a1b2c3d4': 'samdhi'})
assert ol.concept('c') is not None and ol.concept('c-sharp') is None
```
The real run moves the concepts and every slug that points to them. An
answer that is the text `c` stays as it is. A second run has nothing to
do.
``` python
test_eq(ol.reslug(), {'c': 'c-sharp', 'a1b2c3d4': 'samdhi'})
test_eq((ol.concept('c'), ol.concept('a1b2c3d4')), (None, None))
test_eq((ol.concept('c-sharp').name, ol.concept('samdhi').statement), ('C#', 'Sounds change where words meet.'))
r = ol.concept('ranking')
test_eq((r.prereqs, r.related), (['c-sharp'], ['samdhi', 'sa-term-karaka']))
a = ol.attempts('c-sharp')[0]
test_eq((a.concept, a.answer), ('c-sharp', 'c'))
s = ol.session('s1')
test_eq((s.concepts, s.log[0]['concept'], s.focus), (['c-sharp'], 'c-sharp', 'samdhi'))
test_eq(ol.reslug(), {})
```
`mark_attempt` changes a saved attempt. `add_gap` keeps each gap one
time, and `graph` shows it as a `gap` link.
``` python
gl = temp_learning()
gl.put('Functions and the neuron', 'z = w . x + b.', 'ml'); gl.put('Weighted sum', 'Add each input times its weight.', 'ml')
a = gl.record('functions-and-the-neuron', 'Why does z double?', answer='2z')
test_eq(gl.mark_attempt(a.id, taught=True)['taught'], True)
test_eq(gl.attempts('functions-and-the-neuron')[0].taught, True)
gl.add_gap('functions-and-the-neuron', 'weighted-sum'); gl.add_gap('functions-and-the-neuron', 'weighted-sum')
test_eq(gl.concept('functions-and-the-neuron').gaps, ['weighted-sum'])
assert dict(source='weighted-sum', target='functions-and-the-neuron', kind='gap') in gl.graph()['links']
```