This module is the base of the engine. Every other module imports from it, and it imports from no other module of abhyasa. It holds what abhyasa knows about the learner, and the shelf where it keeps it. The parts come in the order that they import each other:
Two functions of the profile ask a model. They import the model layer (abhyasa.models) when they run, because that layer imports this module.
Import the modules that this module uses. The heavy ones load inside the functions that need them, so the import stays light.
One table for each value that you can change, and the file that keeps your changes.
Each value that a person can change is a row in one table, SETTINGS. A row has a group, a name, a label, one line of help, a type, a default and, for a number, a range. The engine, the web app and the Settings page all read this table. Thus each value has one place.
Your changes go in settings.json in cfg_dir(). The code reads a value when it uses it, so a change works at once, with no restart. A saved value outside its range gets the nearest end of the range. A saved value of the wrong type gets the default. A bad file never stops the app.
This module imports only the standard library and fastcore. Each engine module reads its values from here, so the import must stay light.
The Settings page shows the groups in this order. Each group has a label and one lead line.
add puts one row in SETTINGS. The name is section.key: the section of settings.json and the key in it. The type comes from the default. more puts the row under the closed “More” part of its group. store='profile' marks a value that the vault keeps in your profile, not this file. hi can be the name of another row. Then that value is the top of the range. ui=False marks a value that its group shows with its own controls. reset=False keeps a value when you use the defaults of its group: the theme, and the speech engine and model.
Add the setting name to SETTINGS and return its row.
The Look group keeps its own controls: the theme swatches and the mode. The Learning group holds the three values of your profile.
The Walks group sets the size of a walk and of a side walk.
The Talks group sets how a talk runs: the questions, the time, the vote and the room.
The Checks and grading group sets the tries, the grade of a shaky pass, and when an idea counts as learned.
The Review group sets the time to the next review. A daily limit on review is new: 0 means no limit, as before.
The Links group sets how much of a link the tutor gets. The pages read by default can not be more than the page cap.
The Research group sets the web search for a step with no source. The tutor searches the web for the idea, reads the best pages, and quotes one passage word for word. When research is off, the tutor does no web search, and the step says that the text is in the tutor’s words.
The Speech to text group keeps the engine and the model, and the pauses at which the mic cuts what you say. The Screen guide group has the voice and the look of a point mark. A point mark is weaker than a circle in each look, so a circle is always the main mark of a step. The default is tick, because it has no ring that looks like a circle.
The Speaking voice group picks who reads the steps of the screen guide aloud. pocket runs Pocket TTS on this Mac. openai sends each line to OpenAI. browser is the voice of the browser (or of the Mac in the overlay). The voice module reads these values. The default is pocket. Until its model is downloaded, the browser speaks. An old hf or kokoro value is not one of the choices, so it reads as pocket. The speed is only for OpenAI: Pocket TTS has no speed.
The Models group has the default model and one model for each task. An empty value uses the default model. The tasks after extract borrow the model of an older task when they have none: BORROWS says which. The screen guide (guide) and its figure labels (guide_vision) use the model for hints until you pick one. The reply caps are in tokens.
The Lists group sets how many items show in the lists of the app. All of it is under “More”.
Each group except Prompts has rows, and each row is in a group. The Prompts group lists the prompts of prompts.PROMPTS.
DFLT_SETTINGS is the default of each section: the rows that the file keeps, the saved prompt texts and the models that the Settings page downloaded. settings() gives the saved settings over the defaults. A section that is not a dict in the file is ignored, and a top-level key that is not known is dropped.
The folder for abhyasa’s settings and keys: $ABHYASA_CONFIG, else ~/.config/abhyasa.
_saved reads the file only when it changed, so a value read in a loop costs one stat.
save_settings writes the file: first a temporary file with its own name, then a rename, so a crash never leaves half a file. A lock keeps two changes from the web app apart. It does not keep a value that is the default. update_settings changes keys in one section.
Change some keys in one section of the settings.
With no file, settings() gives the defaults. A change in one section keeps the others.
A file from before this table still loads: its sections update the defaults, and the new sections get their defaults.
_path().write_text(json.dumps(dict(stt=dict(engine='mlx', model='m', segment=True), models=dict(question='', grade='', hint='', lesson=''),
tokens=dict(question='', grade='', hint='', lesson='', extract=''), look=dict(theme='ink', mode='dark'), prompts={}, downloaded=[])))
d = settings()
test_eq((d['stt']['engine'], d['stt']['pause'], d['look']['theme'], d['walks']['steps'], d['models']['extract']), ('mlx', 700, 'ink', 7, ''))
save_settings(dict(prompts={}, downloaded=[]))A file that is not JSON, or a section that is not a dict, gives the defaults. An unknown top-level key is dropped.
fit makes a saved value fit its row. A number outside the range gets the nearest end. A value of the wrong type gets the default. A whole number can come as 12.0 from a form; it stays a whole number. A choice must be one of the choices.
The value v made to fit row r: clamped to the range, or the default for the wrong type.
The top of the range of row r: a number, or the value of the row that r.hi names.
val gives the value of a row now: the saved value made to fit, else the default. The engine calls it each time it needs the value.
sources.roots is a text list, one path for each line. A source folder has the same rules as a folder that you name: it must be in your home folder, it must not be the home folder itself, and no part of it can be hidden or a link. source_root_why gives the reason when a path breaks a rule. roots gives the paths that keep the rules, and dropped_roots gives the others with their reasons, so the Settings page can show them.
The lines of sources.roots that abhyasa does not use, each as (line, reason).
The paths of the setting sources.roots that keep the rules of source_root_why, one for each line that is not empty.
Why abhyasa does not read the source folder p: outside the home folder, the home folder itself, hidden, or a link. ’’ when it can.
A saved 50 ideas becomes 7, the most. A string gives the default. A share above 1 becomes 1.
The top of the pages read by default is the page cap.
The look of a point mark is one of three choices. A look that is not a choice gives the default, tick.
set_val saves a value made to fit its row, and returns it. reset puts back the default of one row, and reset_group of each row in a group. changed is true when a value is not its default. clamped gives the value in the file when it was outside the range, so the page can say so.
The number in the file for name when it was outside the range, else None.
Use the default of each row of group that the file keeps and that a group reset can change. Return their names.
Save the value v of the setting name, made to fit its row. Return the value in use.
The source folders drop the root of the disk, the home folder itself and a hidden folder.
A value saved out of range is saved as the end of the range. reset and reset_group put back the defaults.
test_eq(set_val('walks.steps', 99), 7)
test_eq(set_val('walks.steps', 1), 3)
test_eq(raw('walks.steps'), 3)
assert changed('walks.steps') and not changed('walks.note')
reset('walks.steps')
test_eq(val('walks.steps'), 7)
set_val('checks.tries', 5); set_val('checks.near', 8)
test_eq(sorted(reset_group('checks')), ['checks.near', 'checks.tries'])
test_eq((val('checks.tries'), val('checks.near')), (3, 7))
update_settings('walks', steps=50)
test_eq(clamped('walks.steps'), 50)
# a number typed by hand as text works, and text that is not a number gives the default
update_settings('tokens', hint='900', grade='many')
test_eq((val('tokens.hint'), val('tokens.grade')), (900, 2048))
# a value that the profile keeps has no reset here
test_fail(lambda: reset('learning.pace'), contains='profile')
save_settings(dict(prompts={}, downloaded=[]))The speaking engine is browser until you change it, and a value that is not an engine gives the default.
Some values keep you safe, so they have no setting: the link checks for local addresses, plain HTTP, the link kinds, the HTML allowlist, the check of a pane address, the local-only hosts, the test allowlist, the grade scale and the depth ladder. The room for phones keeps its safety limits fixed too: the bind rule, the join code, the rate limits, the body size and the headers. FIXED names them, and no row may have one of these words in its name.
The instruction text of the tutor. You can change it on the Settings page.
Each prompt of the tutor has two parts. The variable parts (the concept, the thread, the JSON shape) stay in code. The instruction text is in this registry. Each prompt has a name, a plain label, a note on what it changes, and its default text from code. The Settings page shows each prompt, and you can change its text there. The tutor reads each prompt when it calls the model. Thus a change works on the next question, with no restart.
Import the tools that this module uses. The settings come from settings only when a prompt is read, so the import stays light.
register adds a prompt to PROMPTS and returns its default text. A check can refuse a bad text: it raises ValueError with a plain message.
Add the prompt name with a plain label, a note on what it changes, its dflt text and an optional check. Return dflt.
lines gives the lines of a text that are not empty. need_lines makes a check that refuses a text without exactly n such lines.
A check that refuses a text that does not have n lines that are not empty.
The lines of text that are not empty, without spaces at the ends.
prompt gives the saved text of a prompt, else its default. set_prompt saves a text in the settings, under the prompts key. An empty text, or a text that is the same as the default, removes the saved text. Then the default is in use again, also when a new version changes the default.
Save the text of the prompt name. An empty text uses the default. A text that fails the check raises ValueError. Return the text in use.
The text of the prompt name now: the saved text, else the default.
A small prompt with a check of 2 lines. Its default is in use at first.
A saved text is in use at once, and the settings keep it under prompts.
A text that fails the check is refused with a plain message, and the saved text stays.
An empty text, or only spaces, removes the saved text, so the default is in use again. A text that is the same as the default is not kept.
An unknown name is an error. Then the test prompt goes away.
The learning shelf: concepts, attempts and sessions as vault documents.
This module keeps all learning data in your vault, on one shelf named learning. Each concept, attempt and session is one vault document. Thus each tool that reads the vault can search it. The other modules read and write the shelf only through the Learning class in this module.
Import the tools that this module uses, and the Vault class of the vault.
Search reads the text of a document. The meta JSON keeps the state. Vault.set_meta changes the state and does not embed the text again. The table gives the source URI of each kind of document.
| kind | source |
|---|---|
concept |
learning://concept/<slug> |
attempt |
learning://attempt/<id> |
session |
learning://session/<id> |
These constants name the shelf, give the start state of a concept, and list the fields that merge.
The meta of the document row r: a dict from vishalakshi by_source, or JSON text from t.docs.
slugify makes the key of a concept from its name. It changes a name in an Indic script, such as Devanagari, to IAST first. It changes +, # and & to words, so C, C++ and C# get different slugs. A name with no letters that it can read gets a short hash of the name. Thus the same name always gives the same slug. _src makes the source URI of a document.
Whether name has two characters at least, and a letter or a digit. A name such as x is not an idea.
Whether concept c shows in Library and the graph: an idea_name, and not a statement of one word. The stray x of an old tool had the statement x.
Whether a new concept name with statement and no source text is an idea: an idea_name, and a statement of 3 words or more. A plan or a gap with no source uses it.
Whether name can name a new idea: it has a letter or a digit, and it is not one lower-case letter such as x. One capital letter, such as C or R, can.
Make a lower-case ASCII slug from s (vishalakshi slug, whole): saṃdhi and संधि give samdhi, and C++ gives c-plus-plus.
_union merges two lists and keeps the order. concept_text makes the text that search reads.
The searchable text of a concept: its name, statement, aliases and subject.
Learning opens one shelf of a vault. It also accepts a path, or None for the default vault.
Concepts, attempts and sessions on one shelf of your vault.
A vault file holds more stores than the shelves of documents. For example, the vault keeps the names that it finds in a store named entities. That store has no doc_id column and no docs table. vishalakshi’s Vault.shelf_names gives only the shelves that hold documents. doc_shelf opens one of them with shelf(create=False), or gives None. It never makes a new shelf, so a name in a link or in an old walk does not add a store to your vault.
The shelf of documents name (vishalakshi shelf(create=False)), or None when the vault has no such shelf. It never makes a shelf.
Each method returns concepts as AttrDicts with the learner state filled in.
All concepts, or the concepts of one subject.
The concept that a search hit (its doc_id) belongs to, or None when the hit is not a concept.
One concept by slug, with its learner state.
put adds a concept or adds to it. It keeps the learner state, merges the list fields and replaces the other fields. It does not change the group of a concept that the learner moved by hand (see Groups below). Two fields say what the learner must understand. core is the understanding in 1 or 2 sentences: what the idea does and why. It has no sizes, years or scores, unless the number is the idea. wrong lists 1 to 3 common wrong ideas, each as {'think': ..., 'fact': ...}. The question and the grade work from these two fields. statement stays as the reference text. An empty core or wrong keeps the old value. wrong_ideas makes each wrong idea a dict. clean_name makes a name one line with no control characters, at most NAME_LEN characters. put and rename use it.
def put(
name:str, # the concept's name
statement:str='', # a true, self-contained statement of the idea
subject:str='', # e.g. 'sanskrit', 'jyotisha', 'ml'
slug:str=None, # defaults to `slugify(name)`
prereqs:tuple=(), # slugs this concept needs first
related:tuple=(), # slugs linked to it, in any direction
aliases:tuple=(), # other names
provenance:tuple=(), # where it came from: dicts such as `{'src': 'pasted', 'at': ...}`
questions:tuple=(), # ready questions: `{'q':..., 'a':..., 'depth':...}`
core:str='', # the understanding: what the idea does and why, in 1 or 2 sentences
wrong:tuple=(), # 1 to 3 common wrong ideas: `{'think':..., 'fact':...}`
**meta
)->fastcore.basics.AttrDict:Add a concept or add to one. Keep the learner state, merge the list fields and replace the other fields. A name that you set by hand stays.
At most n wrong ideas from xs, each as {'think': str, 'fact': str}. A plain string is a think with no fact; an empty one goes.
s as one line of at most n characters, with no control or format characters. The joiners of Indic scripts stay.
update and link change one concept. drop removes it.
Remove a concept. Its attempts stay as a record of what happened.
Change the state of a concept and do not embed its text again.
abhyasa 0.0.5 and older made slugs in a different way. A name in Devanagari got a random slug, and C, C++ and C# got the same slug, c. reslug finds each concept that has an old slug. It moves the concept to the slug that slugify gives now. It also changes the slug in each place that points to it: the links of other concepts, attempts, sessions, walks and demos. A slug that a source sets, such as sa-term-karaka, does not change.
reslug finds the moves first. With dry_run it stops there and writes nothing.
A concept can sit in a group under its subject. For example, Magha sits at jyotisha › nakshatras › Magha. The path is a list of at most two short names in the group field. group_by tells who set it: hand for the learner, model for the tutor. The tutor never moves a concept that the learner moved by hand. An old vault has no groups, so each concept shows under “Not sorted yet”.
clean_group makes a path safe to keep and to show. Each name has only letters, digits, spaces and hyphens, and at most GROUP_LEN characters.
path as at most GROUP_DEPTH short plain names. A text splits at /, > or ›. Empty parts, repeats and a first part that is the subject drop.
set_group moves one concept. A move by hand marks it. A move by the model does not change a concept with the mark. An empty path by hand also keeps the mark: the learner chose no group, so the tutor does not sort the concept again. groups lists the paths in use under one subject, with their parents, so a new concept can go in a group that is there.
Place a new concept in the group path, as the tutor. One step under a lock: it looks at the concept again, and changes nothing when the concept is gone, has a group, was kept out of the groups by hand, or was undone. Return whether it placed the concept.
Undo a placement by the tutor: no group, and the tutor does not place the concept by itself again. A group that you set, or that a saved sort set, stays.
The concepts that the tutor may place: no group, and not kept out of the groups by hand.
The group paths of subject: the kept ones and the ones in use, each one time and with its parents, sorted.
Keep the group paths of subject: groups (default: the ones in use) added to the ones kept before. A kept group is never dropped by the tutor. With replace, the list becomes groups and the hand groups. Return the list.
The group paths of subject that you made or renamed. The tutor never renames or drops them.
The group paths kept for subject when a sort was saved; None when none was.
Put concept slug in the group path, set by the learner (hand) or the tutor (model).
You can sort the concepts yourself. rename gives a concept a new name and marks it named_by='hand', so a new extract never renames it. The old name stays as an alias, so the find box still finds it. rename_group gives a group a new name. Each concept in the group, or in a group below it, gets the new name and group_by='hand'. A name that is the name of a group beside it, in any case and without marks, joins the two groups. new_group adds an empty group. hand_groups lists the groups that you made or renamed. “Sort everything again” keeps them. snapshot and restore are the Undo of one change by hand: snapshot keeps the fields that a change by hand can touch, and restore writes them back.
Write back the fields of snap (from snapshot). A concept that is gone stays gone. Return the slugs written.
The fields that a change by hand can touch, for the concepts slugs and the group lists of subjects, so restore can write them back.
Take the hand mark off the groups paths of subject, so the tutor can sort them again. Return the hand groups.
Add the empty group path to subject, made by hand. Return the group list.
Give the group path of subject the last name name, set by hand. Its concepts and the groups below it move with it. A name of a group beside it joins the two. Return the slugs changed.
Give concept slug the name name, set by hand: a new extract never renames it. The old name becomes an alias.
Whether the group g is the group path or a group below it. Case and marks do not count.
s in lower case with no marks (litesearch fold_text), so Maghā and MAGHA are one name, and संधि gives samdhi.
set_subject gives a subject to a concept. readable_name and repair_names make a readable name for a concept whose name is its slug.
One time for each shelf, give each concept whose name is its own slug, with hyphens, a readable name. Return the slugs that changed. A name that differs from the slug, or that you set by hand (named_by), stays.
name as words: a name that is a slug with hyphens, or an empty name (words from statement), becomes a short sentence-case name. Else name.
Give concept slug a subject, set by the learner (hand) or the tutor (model). The tutor sets only an empty subject, and never on a concept that you moved by hand.
s as one short plain lower-case name; ’’ when it has none.
tree puts concepts in a tree for the Concepts page: a node for each subject, then its groups, then their groups. A node has its name, its path under the subject, n (the concepts in it and below it), kids (its groups by name) and cs (its own concepts). A concept with no group is in cs of its subject. Two names that differ only in case are one group.
The subject nodes of concepts, sorted by name, each with its groups and the count of concepts below it. groups ({subject: paths}) adds groups with no concept, under a subject that shows.
Each answer is one attempt document. Thus search can find what the learner said. The tutor can also find old questions and not ask them again.
The searchable text of an attempt: the question, the answer, the think-aloud and the feedback.
record saves one attempt. The vault joins two sources with the same text into one document. So record asks for a new document each time (force=True): two attempts with the same words stay two records. Each one has its own source, learning://attempt/<id>.
def record(
concept:str, question:str, answer:str='', thinking:str='', correct:bool=False, confidence:int=None, depth:int=1,
kind:str='explain', hints:int=0, dont_know:bool=False, grade:int=None, feedback:str='', first_error:str='',
misconception:str='', session:str='', **meta
)->fastcore.basics.AttrDict:File one attempt as a vault document, so the learner can search what they said.
attempts reads them back, with the newest first.
Recent attempts, newest first, for one concept or one session; with since, only the attempts from that time on.
mark_attempt adds fields to a saved attempt, for example a lesson that you asked for later. add_gap keeps a gap that you named on a concept.
Record that you said you do not know concept gap while you learned slug.
Merge kv into the meta of attempt aid.
A session summary is one document. A new summary with the same id replaces the old one.
The filed sessions, the most recently changed first; with journey, only the sessions of that walk.
One filed session by id, as its meta.
File a session, and replace an earlier one with the same id. The meta changes also when the summary text does not.
search uses the hybrid search of the vault. Each hit has the kind, title and meta of its document. The method stats, the marker of a repair, a session with no answers and an open question stay out of it. Two attempts with the same text show as one hit. graph gives the nodes and links for the graph view.
Nodes and links (prereq, related, gap) for a graph view. A link to a slug that has no concept is dropped.
Hybrid search over the shelf. Each hit also has the kind, title and meta of its document, so a page can show it by kind. The method stats, the marker of a repair, the history of the changes to a talk or a walk, a session with no answers and an open question stay out: they have no text that you wrote or learned, and an open question has its answer key.
Each test uses a vault in a temporary folder with the offline hashing encoder. The tests never open the real vault of the learner.
put makes a concept with the start state and the slug from its name.
update changes the state.
A concept keeps its core and its wrong ideas. A later put with no core keeps them. A wrong idea as a plain string becomes a dict.
wl = temp_learning()
c = wl.put('Token embeddings', 'Each token ID picks a row of a learned matrix.', 'ml', core='Each token gets a learned vector, so tokens used in the same way end up close.',
wrong=['the ID number itself carries meaning', dict(think='the vectors are fixed', fact='training changes them')])
test_eq(c.wrong, [dict(think='the ID number itself carries meaning', fact=''), dict(think='the vectors are fixed', fact='training changes them')])
test_eq(wl.put('Token embeddings', 'Rows of E.').core, c.core)A second put keeps the learner state and merges the lists, also when the text changes.
concepts and subjects filter by subject. graph keeps only the links between known concepts.
lr.put('Ranking', 'Order items by a score.', 'ml')
lr.put('Sandhi', 'Sounds change where words meet.', 'sanskrit')
test_eq(len(lr.concepts('ml')), 2)
test_eq({r['subject'] for r in lr.subjects()}, {'ml', 'sanskrit'})
g = lr.graph()
test_eq(g['links'], [dict(source='ranking', target='reciprocal-rank-fusion', kind='prereq')])record saves an attempt. attempts and search find it again.
a = lr.record('reciprocal-rank-fusion', 'Why does RRF ignore raw scores?', answer='Because scores from different systems are not comparable.',
thinking='bm25 and cosine have different scales', correct=True, confidence=80, depth=3)
test_eq(lr.attempts('reciprocal-rank-fusion')[0].id, a.id)
test_eq(lr.attempts('ranking'), [])
test(lr.search('scales comparable', kind='attempt'), [], lambda a,b: len(a) > 0)drop removes a concept.
A real vault also keeps a store of names, entities, with no doc_id column. shelf_names leaves it out. It keeps the main shelf, the learning shelf and a shelf of notebooks.
dv = temp_learning()
dv.vault.shelf('notebooks').add('Notes on softmax.', title='nbs/softmax', source='/x/softmax.md', kind='md')
dv.vault.db.get_graph('store', ann=False) # the store of names, as vishalakshi makes it
assert 'entities' in dv.vault.db.table_names() # the store of names is in the vault, though vishalakshi 0.1.36 no longer lists it as a shelf
test_eq(sorted(dv.vault.shelf_names()), ['learning', 'notebooks', 'store'])shelf_names reads only the names of the stores. It does not count the documents of each store, so SQLite logs no error for a store of names with no docs table. A page can call it many times.
class _Seen(logging.Handler):
def __init__(self): super().__init__(); self.got = []
def emit(self, r): self.got.append(r.getMessage())
seen, root = _Seen(), logging.getLogger()
root.addHandler(seen)
try: test_eq(sorted(dv.vault.shelf_names()), ['learning', 'notebooks', 'store'])
finally: root.removeHandler(seen)
test_eq([m for m in seen.got if 'no such table' in m], [])doc_shelf opens a shelf of documents. It gives None for the store of names and for a name that is not in the vault, and it does not make that shelf.
The search fetches more hits until it has limit real hits, or the shelf has no more. Here 30 sessions with no answers match the words best. Three concepts match only one word. All three concepts come back.
lr.log_session('s1', '1 of 2 passed.', start=1., updated=2., asked=2)
lr.log_session('s1', '1 of 2 passed.', start=1., updated=3., asked=2, ended=3.) # the same text: the meta still changes
test_eq((lr.session('s1').updated, lr.session('s1').ended, lr.session('s1').summary), (3., 3., '1 of 2 passed.'))
test_eq(len(lr.sessions()), 1)sessions lists the most recently changed session first.
With journey, sessions reads only the sessions of that walk, in the store.
search leaves out a session with no answers and the method stats. The session is still filed, so you can continue it. Each hit names its kind.
lr.log_session('s4', '0 of 0 passed. Concepts: none.', start=7., log=[])
lr.v.add('How you learn: method counts, passed.', title='How you learn', source='stats:test', kind='stats')
hs = lr.search('passed', limit=20)
test_eq([h for h in hs if h['kind'] == 'stats' or h['meta'].get('id') == 's4'], [])
test_eq('s1' in [h['meta'].get('id') for h in hs if h['kind'] == 'session'], True) # a session with answers stays
test_eq(lr.session('s4').id, 's4')The search fetches more hits until it has limit real hits, or the shelf has no more. Here 30 sessions with no answers match the words first, and three concepts match them too. All three concepts come back.
concept_by_doc gives the concept of a search hit. It gives None for a hit that is not a concept.
cl = temp_learning()
cl.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank).', 'ml')
cl.record('reciprocal-rank-fusion', 'Why ranks?', answer='scales differ')
test_eq(cl.concept_by_doc(cl.search('rank fusion', kind='concept')[0]['doc_id']).slug, 'reciprocal-rank-fusion')
test_is(cl.concept_by_doc(cl.search('scales differ', kind='attempt')[0]['doc_id']), None)slugify reads Devanagari and Telugu, keeps C, C++ and C# apart, and gives the same hash slug for the same unreadable name.
This test makes a vault for reslug as abhyasa 0.0.5 left it: C# saved as c, a Devanagari name with a random slug, and links, an attempt and a session that use these slugs.
ol = temp_learning()
ol.put('C#', 'A language from Microsoft.', 'code', slug='c')
ol.put('संधि', 'Sounds change where words meet.', 'sanskrit', slug='a1b2c3d4')
ol.put('karaka term', 'A set slug.', 'sanskrit', slug='sa-term-karaka')
ol.put('Ranking', 'Order items by a score.', 'ml', prereqs=['c'], related=['a1b2c3d4', 'sa-term-karaka'])
ol.record('c', 'What is C#?', answer='c')
ol.log_session('s1', 'One question.', concepts=['c'], log=[dict(concept='c', question='What is C#?')], focus='a1b2c3d4')A dry run lists the moves and changes nothing. A slug that a source sets is not in the list.
The real run moves the concepts and every slug that points to them. An answer that is the text c stays as it is. A second run has nothing to do.
test_eq(ol.reslug(), {'c': 'c-sharp', 'a1b2c3d4': 'samdhi'})
test_eq((ol.concept('c'), ol.concept('a1b2c3d4')), (None, None))
test_eq((ol.concept('c-sharp').name, ol.concept('samdhi').statement), ('C#', 'Sounds change where words meet.'))
r = ol.concept('ranking')
test_eq((r.prereqs, r.related), (['c-sharp'], ['samdhi', 'sa-term-karaka']))
a = ol.attempts('c-sharp')[0]
test_eq((a.concept, a.answer), ('c-sharp', 'c'))
s = ol.session('s1')
test_eq((s.concepts, s.log[0]['concept'], s.focus), (['c-sharp'], 'c-sharp', 'samdhi'))
test_eq(ol.reslug(), {})mark_attempt changes a saved attempt. add_gap keeps each gap one time, and graph shows it as a gap link.
gl = temp_learning()
gl.put('Functions and the neuron', 'z = w . x + b.', 'ml'); gl.put('Weighted sum', 'Add each input times its weight.', 'ml')
a = gl.record('functions-and-the-neuron', 'Why does z double?', answer='2z')
test_eq(gl.mark_attempt(a.id, taught=True)['taught'], True)
test_eq(gl.attempts('functions-and-the-neuron')[0].taught, True)
gl.add_gap('functions-and-the-neuron', 'weighted-sum'); gl.add_gap('functions-and-the-neuron', 'weighted-sum')
test_eq(gl.concept('functions-and-the-neuron').gaps, ['weighted-sum'])
assert dict(source='weighted-sum', target='functions-and-the-neuron', kind='gap') in gl.graph()['links']Two attempts with the same text are two records. The vault does not join them, so each one counts in your history and in the misses that the tutor counts. A field added later goes on the right attempt.
ld = temp_learning(); ld.put('Ranking', 'Order items by a score.', 'ml')
a1, a2 = [ld.record('ranking', 'What is a rank?', 'a score', feedback='Look at positions.', final=True) for _ in range(2)]
test_eq(ld.attempts('ranking').attrgot('id'), [a2.id, a1.id])
ld.mark_attempt(a2.id, taught=1)
test_eq([a.get('taught') for a in ld.attempts('ranking')], [1, None])A search shows the same attempt text only once.
An old vault has no groups. A concept loads with an empty path, and no one set it.
clean_group keeps at most two short plain names. It drops marks, empty parts, a repeat and a first part that is the subject. A text splits at /, > or ›.
test_eq(clean_group(['Nakshatras', 'Padas', 'More']), ['Nakshatras', 'Padas'])
test_eq(clean_group(' nakshatras / padas! '), ['nakshatras', 'padas'])
test_eq(clean_group(['Nakshatras!', ' deva gana? ']), ['Nakshatras', 'deva gana'])
test_eq(clean_group(['jyotisha', 'nakshatras'], 'jyotisha'), ['nakshatras'])
test_eq(clean_group(['', 'Nakshatras', 'nakshatras', 3]), ['Nakshatras'])
test_eq(clean_group('saṃdhi › स्वर'), ['saṃdhi', 'स्वर'])
test_eq(len(clean_group(['x' * 90])[0]), GROUP_LEN)
test_eq((clean_group(None), clean_group({'a': 1}), clean_group('')), ([], [], []))set_group by the model moves a concept. A move by hand sets group_by='hand'. After that, a move by the model changes nothing.
A new put with a group, as from a new source, does not move a concept that you moved by hand. It keeps the learner state.
An empty path by hand takes the concept out of its group and keeps the mark. So the tutor does not sort it again, and a move by the model changes nothing.
groups lists the paths in use under one subject, with their parents. A path in another subject does not show.
set_subject gives a subject to a concept that has none. A move by the tutor never changes a concept that you moved by hand, and it never replaces a subject. A move by hand always works.
sj = temp_learning()
sj.put('Alpha', 'One.'); sj.put('Beta', 'Two.'); sj.put('Gamma', 'Three.', 'ml')
sj.set_group('beta', 'mine')
test_eq(sj.set_subject('alpha', ' Star Maps! ').subject, 'star maps')
test_eq(sj.concept('alpha').subject_by, 'model')
test_eq(sj.set_subject('beta', 'stars').subject, '') # moved by hand: the tutor does not touch it
test_eq(sj.set_subject('gamma', 'stars').subject, 'ml') # it has a subject
test_eq(sj.set_subject('gamma', 'stars', by='hand').subject, 'stars')A name that is its own slug, with hyphens, gets a readable name one time. A name that differs from the slug stays. The empty name of the finder gets words from the statement.
test_eq(readable_name('attention-heads-and-anaphora-resolution'), 'Attention heads and anaphora resolution')
test_eq(readable_name('Softmax'), 'Softmax'); test_eq(readable_name('softmax'), 'softmax'); test_eq(readable_name('my-Note'), 'my-Note')
test_eq(readable_name('', 'A rank is a position in a list. More.'), 'A rank is a position in')
rn = temp_learning()
rn.put('attention-heads-and-anaphora-resolution', 'Heads.', 'ml', slug='attention-heads-and-anaphora-resolution')
rn.put('my-note', 'Mine.', 'ml', slug='note')
rn.put('foo-bar', 'Typed.', 'ml', slug='foo-bar', named_by='hand')
def old_vault(l):
"Forget the mark of the repair, as in a vault from before it."
l.v.forget(first(l.v.t.docs(where='source=?', where_args=['learning://marker/names']))['id'])
old_vault(rn)
test_eq(Learning(rn.vault).concept('attention-heads-and-anaphora-resolution').name, 'Attention heads and anaphora resolution')
test_eq((rn.concept('note').name, rn.concept('foo-bar').name), ('my-note', 'foo-bar')) # a name that differs from the slug, or that you typed, stays
test_eq(rn.repair_names(), [])The repair runs one time: a mark on the shelf says that it ran. A vault that cannot be read or changed does not break the load. A change of name or subject also makes the search text and the title again.
rn.put('late-slug-name', 'Late.', 'ml', slug='late-slug-name')
test_eq(rn.repair_names(), []) # the mark is set: it does not look again
test_eq(rn.concept('late-slug-name').name, 'late-slug-name')
old_vault(rn)
with mock_patch.object(Learning, 'repair_names', side_effect=OSError('read only')): Learning(rn.vault) # the load goes on
test_eq(rn.repair_names(), ['late-slug-name'])
doc = lambda l, s: first(l.v.t.docs(where='source=?', where_args=[f'learning://concept/{s}']))
test_eq(doc(rn, 'late-slug-name')['title'], 'Late slug name')
sj.set_subject('gamma', 'cosmos', by='hand')
test_eq(sj.concept('gamma').subject, 'cosmos')
assert sj.concept('gamma').doc_id in [h['doc_id'] for h in sj.v.search('cosmos', 3)] # the search text has the new subjectThe mark of the repair is a document, but search never gives it.
When you save a sort, the tutor keeps the group list of each subject and does not move a sorted concept again. save_groups keeps the list. groups gives the saved groups and the groups in use. A saved group stays when no concept is in it. unsorted lists the concepts that the tutor may still place: no group, and not set by hand.
cl = temp_learning()
for n in ('Ashvini', 'Bharani', 'Krittika'): cl.put(n, f'{n}.', 'jyotisha')
test_is(cl.saved_groups('jyotisha'), None)
cl.set_group('ashvini', ['Stars'], by='sort')
test_eq(cl.save_groups('jyotisha'), [['Stars']])
cl.set_group('ashvini', [], by='hand'); cl.set_group('bharani', ['Signs'], by='sort')
test_eq(cl.save_groups('jyotisha'), [['Signs'], ['Stars']]) # a kept group stays when no concept is in it
test_eq(cl.groups('jyotisha'), [['Signs'], ['Stars']])
test_eq(cl.saved_groups('JYOTISHA'), [['Signs'], ['Stars']])
test_eq(cl.concept('bharani').group_by, 'sort')A concept with a group is sorted. A concept with no group is not sorted, except when you set it by hand to no group. unplace is the Undo of a placement by the tutor: the concept has no group, and the tutor does not place it by itself again.
test_eq(cl.unsorted('jyotisha').attrgot('slug'), ['krittika']) # ashvini: kept out by hand; bharani: sorted
cl.set_group('krittika', ['Stars'], by='placed')
test_eq(cl.unplace('krittika').group_by, 'undone')
test_eq((cl.concept('krittika').group, cl.unsorted('jyotisha').attrgot('slug')), ([], ['krittika']))
test_eq(cl.unplace('bharani').group, ['Signs']) # only a placement by the tutor can be undone
test_eq(cl.unplace('ashvini').group_by, 'hand')A name with fewer than two characters, or with no letter or digit, is not an idea: usable_name says so, idea_name is the rule for the name of a new concept: a letter or a digit, and not one small letter such as x. One capital letter, such as C, is a name. put refuses a new concept with another name, so no tool, plan or text can file one. A plan or a gap has no source text, so it also needs real_idea: a statement of 3 words or more. Library and the graph leave out a concept that breaks the name rule or has a statement of one word (listed), such as the x that an older tool filed. The marker and the group list never show in search.
assert usable_name('Ab') and usable_name('x2') and usable_name('C#') and not usable_name('x') and not usable_name(' - ') and not usable_name(None)
test_fail(lambda: cl.put('x', 'x', 'code'), contains='not the name of an idea') # one small letter
test_fail(lambda: cl.put('z', 'z is a letter for a value.', 'code'), contains='not the name of an idea')
test_eq(cl.put('C', 'C is a compiled systems language.', 'code').name, 'C') # one capital letter
assert cl.concept('x') is None and listed(cl.concept('c')) and not listed(dict(name='Bias', statement='x'))
test_eq((real_idea('Bias', 'x'), real_idea('C', 'C is a language.')), (False, True)) # with no source text, a statement of 3 words or more
cl.drop('c')
test_eq(cl.search('Groups of jyotisha', kind='groups'), [])
assert not [h for h in cl.search('Groups of jyotisha') if h['kind'] == 'groups']gv.put('Ashvini', 'The first nakshatra.', 'jyotisha', group=['nakshatras', 'deva gana'])
gv.put('Mesha', 'The first rasi.', 'jyotisha', group=['Rasis'])
gv.put('Sandhi', 'Sounds change where words meet.', 'sanskrit', group=['phonology'])
test_eq(gv.groups('jyotisha'), [['nakshatras'], ['nakshatras', 'deva gana'], ['Rasis']])
test_eq(gv.groups('ml'), [])tree puts the concepts under their subject, then their groups. Each node has the count of the concepts in it and below it. A concept with no group is an item of its subject. A group name in another case is the same group.
gv.put('Bharani', 'The second nakshatra.', 'jyotisha', group=['Nakshatras'])
t = tree(gv.concepts())
test_eq([(s.name, s.n) for s in t], [('jyotisha', 4), ('sanskrit', 1)])
j = t[0]
test_eq([(k.name, k.n) for k in j.kids], [('nakshatras', 2), ('Rasis', 1)])
test_eq([c.name for c in j.cs], ['Magha'])
test_eq([(k.path, [c.name for c in k.cs]) for k in j.kids[0].kids], [(['nakshatras', 'deva gana'], ['Ashvini'])])
test_eq([c.name for c in j.kids[0].cs], ['Bharani'])A renamed group takes its concepts with it. They and the group get the hand mark, so the tutor never moves them.
hs = temp_learning()
for n in ('Ashvini', 'Bharani'): hs.put(n, f'{n}.', 'jyotisha', group=['nakshatras'])
hs.save_groups('jyotisha')
test_eq(sorted(hs.rename_group('jyotisha', ['Nakshatras'], 'Lunar mansions')), ['ashvini', 'bharani'])
test_eq({(tuple(c.group), c.group_by) for c in hs.concepts('jyotisha')}, {(('Lunar mansions',), 'hand')})
test_eq((hs.hand_groups('jyotisha'), hs.groups('jyotisha')), ([['Lunar mansions']], [['Lunar mansions']]))
assert not hs.unsorted('jyotisha') and not hs.place('ashvini', ['Other'])A name that is the name of a group beside it, in another case, joins the two groups. The group below a renamed group moves with it.
hs.put('Pada', 'A quarter.', 'jyotisha', group=['Stars', 'padas']); hs.put('Pada two', 'A quarter.', 'jyotisha', group=['Stars x'])
hs.rename_group('jyotisha', ['Stars x'], 'STARS')
test_eq(hs.concept('pada-two').group, ['Stars'])
hs.rename_group('jyotisha', ['Stars'], 'Asterisms')
test_eq((hs.concept('pada').group, hs.concept('pada-two').group), (['Asterisms', 'padas'], ['Asterisms']))
assert ['Stars'] not in hs.groups('jyotisha') and ['Stars x'] not in hs.groups('jyotisha')
test_fail(lambda: hs.rename_group('jyotisha', ['Asterisms'], '!!!'), contains='letters or digits')new_group adds an empty group that stays. save_groups(replace=True) makes the list again and keeps the hand groups. unmark_groups takes the mark off.
hs.new_group('jyotisha', 'Rasis')
assert ['Rasis'] in hs.groups('jyotisha') and ['Rasis'] in hs.hand_groups('jyotisha')
test_eq(hs.save_groups('jyotisha', [['Grahas']], replace=True), [['Asterisms'], ['Grahas'], ['Lunar mansions'], ['Rasis']])
assert ['Asterisms', 'padas'] in hs.groups('jyotisha') # a group in use still shows
hs.unmark_groups('jyotisha', [['rasis']])
assert ['Rasis'] not in hs.hand_groups('jyotisha') and ['Rasis'] in hs.groups('jyotisha')A name that you set stays when a new extract puts the concept again. The old name becomes an alias.
hs.put('Self-attention', 'x', 'ml'); c = hs.rename('self-attention', 'Attention over one sequence')
test_eq((c.name, c.named_by, c.aliases), ('Attention over one sequence', 'hand', ['Self-attention']))
test_eq(hs.put('Self-attention', 'y', 'ml').name, 'Attention over one sequence')
test_fail(lambda: hs.rename('self-attention', ' x '), contains='two letters')
test_eq(hs.rename('self-attention', 'Self\x00 at\x1b[31mtention\n' + 'x' * 200).name, 'Self at [31mtention ' + 'x' * 100) # no control characters, and a cap
hs.rename('self-attention', 'Attention over one sequence')
test_eq(hs.put('Bad\x07 name', 'x', 'ml').name, 'Bad name')snapshot and restore undo one change by hand: the groups of the concepts, their names and the group list.
snap = hs.snapshot(['pada', 'self-attention'], ['jyotisha'])
hs.rename_group('jyotisha', ['Asterisms'], 'Stars again'); hs.rename('self-attention', 'Self focus'); hs.new_group('jyotisha', 'Extra')
test_eq(sorted(hs.restore(snap)), ['pada', 'self-attention'])
test_eq((hs.concept('pada').group, hs.concept('self-attention').name), (['Asterisms', 'padas'], 'Attention over one sequence'))
assert ['Extra'] not in hs.groups('jyotisha') and ['Asterisms'] in hs.saved_groups('jyotisha')
snap = hs.snapshot([], ['ml']); test_is(hs.saved_groups('ml'), None)
hs.new_group('ml', 'Heads'); hs.restore(snap)
test_is(hs.saved_groups('ml'), None) # Undo takes away a list that was not thereThe learner model: how deep you know each concept, when to review it, and what to ask next.
This module holds the learner model as plain functions with no model calls and no storage. It finds what an answer means, updates the state of a concept, finds the next depth and picks the next concept. The tutor calls these functions and the store keeps their results.
Import the tools that this module uses.
Each concept has a depth from 0 to 5. The tutor asks each question at one depth.
| depth | name | a question at this depth asks you to |
|---|---|---|
| 0 | unseen |
(no question) |
| 1 | recognise |
pick the correct statement from options |
| 2 | recall |
state the idea or the rule |
| 3 | explain |
say why it is true, in your own words |
| 4 | apply |
use it to solve a problem |
| 5 | transfer |
use it in a new case or link it to another subject |
The floor is the highest depth that you pass reliably. The ceiling is the lowest depth that you failed (6 until you fail). The next question goes between them. Thus each answer moves one edge of this range.
These constants name the depths, give the question kind at each depth, and set the goal and the high-confidence line.
Correctness and confidence together tell what an answer means.
| high confidence | low confidence | |
|---|---|---|
| correct | solid |
fragile |
| wrong | misconception |
gap |
“I don’t know” is a gap. A correct answer that used two or more hints is fragile.
One of solid, fragile, misconception or gap. With no confidence, only the answer and the hints count.
A correct answer is solid with high confidence. Two hints or low confidence make it fragile.
With no confidence given, a clean pass is solid and a wrong answer is a gap, never a misconception.
A wrong answer is a misconception with high confidence, else a gap. “I don’t know” is always a gap.
Recall falls with time as in FSRS: \(R(t) = (1 + \frac{19}{81}\frac{t}{S})^{-1/2}\). The time \(t\) and the stability \(S\) are in days. At \(t = S\), recall is 90%. Thus the due date is the last review plus \(S\) days.
recall gives the recall now. grow gives the new stability after an outcome.
The new stability after an outcome, from the old stability s and the recall r at the time of the answer.
The chance, from 0 to 1, that you still recall concept c now.
Recall is 90% after \(S\) days, and 0 for a concept with no review.
A pass grows stability. A harder recall grows it more. A gap shrinks it.
update returns the fields to change after one answer at depth d.
| outcome | change |
|---|---|
solid |
The floor rises to d. The streak grows. |
fragile |
The floor stays. The concept is shaky at d, so the next question is at d again. |
misconception |
The ceiling falls to d. The floor falls below d. The misconception is kept. |
gap |
The ceiling falls to d. The floor falls below d. |
The state changes for concept c after one answer at depth, with the outcome in kind.
A solid pass at depth 3 on a new concept raises the floor to 3. The review is due in one day (the setting review.solid).
A confident wrong answer at depth 4 lowers the ceiling to 4 and keeps the misconception.
A miss below the floor lowers the floor. A low-confidence pass makes the concept shaky at that depth.
Ask in the middle of the range between floor and ceiling, so one answer cuts the range in half. Ask a shaky concept at the same depth again. When the ceiling is the floor plus 1, ask at the ceiling, because the work is at that edge.
A new concept starts at depth 3 (explain). An open range gives its middle.
A closed range gives the ceiling. Depth 5 is the top. A shaky depth comes first.
pick chooses the concept. Code makes the choice, not the model, so the app can show the reason.
focus concept, when you chose one and did not learn it yet.After two misses in a row on a concept, pick goes back to its weakest prereq that is below the goal.
learned and ready test one concept. _back finds the prereq to go back to. After a demo, it waits for the demo ladder (see below).
Whether each prereq of c that is a known concept is at depth 3 or more.
Whether concept c is at depth goal or more; the default is the setting checks.goal.
pick returns the concept, the depth and the reason.
The next (concept, depth, why), or None when no review is due and the frontier is empty. Concepts in near come first on the frontier. review=False skips the review of due concepts.
understanding reads one concept and tells how well you know it: the floor, the ceiling, the recall now, and the reasons that it is weak. It uses only the state of the concept. Thus a page of all concepts does not need to read the attempts. weak is true when a reason says that you need help now. strong needs depth 3 or more, recall of 80% or more, and no reasons.
How well you know concept c: floor, ceiling, recall, the reasons it is weak, weak, strong, one state, and a score to sort by.
A concept with a misconception and two misses is weak, with both reasons. A deep, fresh concept with no misses is strong.
now = 1e6
u = understanding(dict(slug='a', floor=1, ceiling=3, misses=2, misconceptions=['scores add'], seen=3, stability=5., last=now), now)
test_eq((u.weak, u.strong, u.reasons[:2]), (True, False, ['a misconception: scores add', 'missed the last 2 times']))
s = understanding(dict(slug='b', floor=4, ceiling=6, seen=5, stability=30., last=now - DAY), now)
test_eq((s.weak, s.strong, s.reasons), (False, True, []))
test_eq(understanding(dict(slug='g', floor=0, provenance=[dict(src='gap')]), now).reasons, ['you said you did not know it'])
assert understanding(dict(slug='f', floor=3, seen=4, stability=1., last=now - 30 * DAY), now).reasons[0].startswith('recall is down to')
test_eq([u.state, s.state, understanding(dict(slug='n')).state], ['weak', 'strong', 'unseen'])Three concepts: b needs a. The first pick is a, because b waits for its prereq.
A started concept below the goal comes first. When a is learned, b is ready.
A due review comes before the frontier, at the floor of the concept.
r = pick(cs, now=11*DAY)
test_eq((r[0].slug, r[1]), ('a', 4)) # a is due: a review at its floor
assert 'due for review' in r[2]
# a due concept with two misses goes back to its prereq; the why still says that it is a review, so the daily review limit counts it
r = pick([dict(slug='p', name='P', prereqs=[], floor=0), dict(slug='q', name='Q', prereqs=['p'], floor=2, seen=3, due=1., misses=2)], now=5*DAY)
test_eq(r[0]['slug'], 'p')
assert r[2].startswith('due for review: two misses on Q')Two misses go back to the weakest prereq. A learned concept that is not due gives no question.
After you saw a demo of a concept, after_demo on the concept counts your misses since then (the demo module keeps it). While the demo ladder runs, _back waits: it steps back only after the third miss since the demo. Here, b has 4 misses in all.
The tests below need set_val.
With no demo in play, or with the setting checks.demo_ladder off, two misses step back, as before.
A concept in near comes first on the frontier, before a concept you started elsewhere.
A gap that you named on a concept comes first, at the easiest depth. After you pass it, or after two questions on it, the concept comes back.
nn = dict(slug='neuron', name='Neuron', floor=0, ceiling=6, gaps=['wsum'])
ws = dict(slug='wsum', name='Weighted sum', floor=0, ceiling=6, seen=0)
c, d, why = pick([nn, ws], focus='neuron')
test_eq((c['slug'], d), ('wsum', 1)); assert 'you said you did not know Weighted sum' in why
test_eq(pick([nn, dict(ws, floor=1, seen=1)], focus='neuron')[0]['slug'], 'neuron')
test_eq(pick([nn, dict(ws, seen=2)], focus='neuron')[0]['slug'], 'neuron')The ways you learn: six text ways (code, maths, analogies, worked examples, diagrams and plain words), and demos, ordering, playgrounds, boards and timelines. This module finds which ways work for you, and when to try a new one.
Each question, hint and lesson uses one way to teach: its method. The tutor picks the method before it writes the prompt. So the tag is known, not guessed. When a question closes, its outcome counts for its method: solid 1, shaky 0.5, a miss 0.
The counts give each method a Beta posterior, for each subject and over all subjects. choose uses them in two ways:
explore in the profile). First choose tries a method that got better recently. Else it uses a random draw (a Thompson sample).The counts are in one stats document on the shelf. This document is only a cache: method_stats builds it again from the attempts.
METHODS maps each method to the line that the prompt gets. EXAMPLES_TO maps the profile’s “examples that help” to methods, to set the prior.
The text of each method is a prompt that you can change in Settings (see prompts). way gives the text in use: your saved text, else the line in METHODS.
The text of method m now: your saved text, else the line in METHODS; ’’ for an unknown method.
With no saved text, way gives the line in METHODS.
stats reads the stats document. It gives empty counts when there is no document. set_stats writes the document. A lock makes sure that two closes at the same time do not lose a count.
The cached stats: method counts for each subject (* is all), recent credits, and calibration.
credit adds one outcome to the counts and returns the new stats. It keeps the last 8 credits of each method, to find a method that gets better.
Count outcome kind for method with the weight w in the stats document.
The stats st with one outcome kind added for method with the weight w, in subject and in *. Only a full outcome (w=1) goes in the recent credits.
Solid adds 1 to a. A miss adds 1 to b. Shaky adds half to each. An unknown outcome or method changes nothing.
st = credit(credit(credit(EMPTY, 'code', 'ml', 'solid'), 'code', 'ml', 'fragile'), 'maths', 'ml', 'gap')
test_eq(st['methods']['ml'], {'code': [1.5, .5], 'maths': [0., 1.]})
test_eq(st['methods']['*'], st['methods']['ml'])
test_eq(st['recent'], {'code': [1., .5], 'maths': [0.]})
test_eq(credit(st, 'code', 'ml', 'closed'), st)The row under a demo or a lesson also counts for the way of that demo or lesson, with a small weight from FEEDBACK. “Not clear” adds 0.25 of a miss, so four of them count as one miss, and one click does not decide the way. “Just right” adds 0.1 of a pass. “Too simple” adds nothing: the way worked, but the step was too small. A part of an outcome does not go in the recent credits, so it does not make a way look better lately.
posterior gives (a, b) for a method: Beta(1, 1), plus 2 on a for a method in the profile’s examples. For a subject, it adds the subject counts and the * counts. It scales the * counts down to at most 4 pseudo-counts. So a new subject starts from what works for you in other subjects.
(right, n) over the last 5 credits of m, when they are at least 0.2 above its mean over all; else None.
The Beta (a, b) of method m in subject.
With no counts, the prior decides. improving finds a method that got better recently.
test_eq(posterior(EMPTY, 'code'), (1., 1.))
test_eq(posterior(EMPTY, 'code', prior=['code']), (3., 1.))
gm = dict(methods={'*': {'maths': [2., 4.]}}, recent={'maths': [0., 0., 1., 1., 1., 1.]})
test_eq(improving(gm, 'maths'), (4, 5))
test_is(improving(dict(gm, recent={'maths': [1., 0., 0., 0.]}), 'maths'), None)choose returns (method, note). The note goes in the line that says why you get this question.
The method for the next question, and a note on why, from the allowed ways (default TEXT_WAYS). A weak concept gets the best method; else, sometimes, another one.
A weak concept gets the best method. With explore='never', choose always gives the best method. With sometimes, it tries a method that got better at times. It still gives the best method most of the time.
cs = dict(methods={'*': {'code': [7., 2.], 'maths': [2., 4.]}}, recent={'maths': [0., 0., 1., 1., 1., 1.]})
m, note = choose(cs, 'ml', weak=True)
test_eq(m, 'code'); assert note.startswith('drill with code: 7 of 9')
test_eq({choose(cs, 'ml', rng=random.Random(i), explore='never')[0] for i in range(30)}, {'code'})
picks = [choose(cs, 'ml', rng=random.Random(i)) for i in range(60)]
assert any(n.startswith('trying maths: you got better') for _, n in picks)
test_eq(sum(m == 'code' for m, _ in picks) > 30, True) # with `sometimes`, the best method still comes most of the timeA session cannot show every way. choose takes allowed: the ways it may pick from. The default is TEXT_WAYS, the ways that every session can show. A list with no known way gives TEXT_WAYS too.
rng = random.Random(0)
assert {'demo', 'order', 'playground', 'board', 'timeline'} <= set(METHODS) and 'way.timeline' in PROMPTS
assert all(choose(EMPTY, rng=rng)[0] in TEXT_WAYS for _ in range(200)) # a session never gets a way it cannot show
assert all(choose(EMPTY, rng=rng, allowed=['demo', 'maths'])[0] in ('demo', 'maths') for _ in range(200))
assert choose(EMPTY, rng=rng, allowed=['board'])[0] in METHODS # an allowed list keeps its own ways
assert choose(EMPTY, rng=rng, allowed=['nope'])[0] in TEXT_WAYS # no known way: back to the text wayscheck_for picks the way for one check of one idea. It looks only at the ways of the idea that a walk can show. It does not repeat the way of the last check when it has another. The kind of the check follows the way, and the depth follows the kind. A short check is depth 3 or more: it asks for the idea and why it holds, never for a fact to recall. It is depth 4 when the floor is 3 or more.
A hands-on way of the idea (a key of CHECK_KIND: demo, order or code) comes before its text ways. The model lists a hands-on way only for an idea that fits it: a process, a sequence, or code. A code check is a notebook that runs in Leela. Without this rule, the learner stats and the profile pick a text way almost every time, because the profile names only text ways. Then nearly every check is a short answer.
Only the idea’s own ways can give a hands-on check. An idea with no way that a walk can show gets a short answer in a text way. A walk passes hands=False when it has its share of hands-on checks (see hands_most in journey).
The way, kind, depth and reason for one check of concept c. A hands-on way of the idea comes before its text ways; hands=False leaves the hands-on ways out.
A check takes a way of its idea. An idea with no way that a walk can show gets a text way.
c = AttrDict(ways=['code', 'maths'], floor=0)
k = check_for(c, EMPTY, rng=random.Random(1))
assert (k.way, k.kind, k.depth) in {('code', 'notebook', 4), ('maths', 'short', 3)} and k.why
assert check_for(AttrDict(ways=['demo'], floor=0), EMPTY).kind == 'demo'
assert check_for(AttrDict(ways=['timeline'], floor=3), EMPTY).way in TEXT_WAYS # no timeline view yet
assert check_for(AttrDict(ways=['words'], floor=0), EMPTY).depth == 3 # a new idea: say the idea and why, not a fact
assert check_for(AttrDict(ways=['words'], floor=3), EMPTY).depth == 4
assert all(check_for(AttrDict(ways=['demo', 'code'], floor=0), EMPTY, rng=rng, last='demo').way == 'code' for _ in range(20))An idea that lists a hands-on way (demo, order or code) gets a check of that kind. This is true when a text way comes first in the list, and when the profile likes a text way. An idea with only text ways still gets a short answer.
assert all(check_for(AttrDict(ways=['words', 'example', 'order'], floor=0), EMPTY, random.Random(k), prior=['example']).kind == 'order' for k in range(50))
assert all(check_for(AttrDict(ways=['maths', 'demo'], floor=0), EMPTY, random.Random(k)).kind == 'demo' for k in range(50))
assert all(check_for(AttrDict(ways=['words', 'analogy'], floor=0), EMPTY, random.Random(k)).kind == 'short' for k in range(50))An idea with no ways, or with no way that a walk can show, gets a short answer, never a code check. With hands=False (the walk has its share of hands-on checks), an idea gets a short answer in one of its text ways, else in any text way.
assert all(check_for(AttrDict(ways=w, floor=0), EMPTY, random.Random(k)).kind == 'short' for w in ([], None, ['timeline']) for k in range(50))
assert all(check_for(AttrDict(ways=['code', 'maths'], floor=0), EMPTY, random.Random(k), hands=False).way == 'maths' for k in range(20))
assert all(check_for(AttrDict(ways=['demo'], floor=0), EMPTY, random.Random(k), hands=False).kind == 'short' for k in range(20))methods_text says what works, for a prompt and for the page. method_stats builds the counts again from the attempts, in order.
Count every final attempt with a method again, oldest first, with the feedback on each attempt, and save the counts. Return the stats.
The methods with counts, best first, as one line; empty when nothing is counted.
The counts that credit_method keeps match a rebuild from the attempts.
ml_ = temp_learning(); ml_.put('Ranking', 'Order items by a score.', 'ml')
for m, k in (('code', 'solid'), ('code', 'fragile'), ('maths', 'gap')):
ml_.record('ranking', f'{m} {k}?', final=True, method=m, outcome=k); ml_.credit_method(m, 'ml', k); time.sleep(.01)
inc = ml_.stats()
test_eq(ml_.method_stats()['methods'], inc.methods)
assert methods_text(inc, 'ml').startswith('Ways that work for you: code 2 of 2')What abhyasa knows about you: your background, goals, interests, the examples that help you, your pace, and what confuses you.
The tutor asks better questions when it knows the learner. A coder gets code examples. A Sanskrit reader gets a Sanskrit example. This module keeps one profile document on the learning shelf. The profile gets data in three ways: a short first conversation, edits by the learner, and changes that the tutor makes from evidence. The module keeps each change with who made it and why. So the learner can see each change and undo it. The profile stays in the vault on this machine.
A field is a list of short items, or one value. PACES names the three paces. EXAMPLES names the kinds of example a learner can like.
profile_text gives the profile as lines for a prompt. It is empty when the profile has no data. So the prompt for a new learner does not change.
The profile as prompt lines; empty when nothing is known.
clean makes a value fit its field. A list field takes a list or lines of text. The pace must be a known pace. The session length stays between 3 and 40.
profile reads the document. It uses the defaults for the fields that are not set. known is False until the profile has a saved value.
The learner profile, with its list of changes, newest first.
set_profile changes fields. It keeps each change with the old value, the new value, who made it (you, onboarding or tutor) and why.
Change fields of the profile and keep each change with who made it and why. Return the changes.
undo_profile puts back the old value of one change. It finds the change by its id. Two changes in one save can have the same time, so the time can not tell them apart. A change from before the ids has none, so for it the exact time is used. The undo is a change too, so the list shows all the changes.
Put back the old value of the change ref: its id, or for an older change with no id, its exact time at.
The learner answers four questions, by voice or by text. One model call turns the answers into fields.
Fill the profile from the onboarding answers. Return the changes.
The prompt that turns the onboarding answers into profile fields.
observe reads the recent answers: right or not, confidence, hints, misconceptions and think-aloud. A small model call proposes at most three changes, each with its reason. observe applies them immediately and keeps each one with its reason. So the learner sees them and can undo them.
The prompt that proposes changes to profile p from the evidence ev and the method counts in the stats st.
The last n final answers as short lines for a prompt: the outcome, the method, the first error, gaps you named, lessons you asked for, and what you said.
observe applies the changes. A list field takes add and remove. The pace takes value. The goals and the notes are for the learner only, so observe drops a change to them.
Change the profile from the recent answers. Return the changes, each with its reason.
Each text that explains an idea is for a human, and humans differ. One learner is new to the idea, and another knows it well. A text that only says the definition again teaches little to both. So each teaching prompt gets two more parts:
teach.human, one shared prompt with the rules on how to teach. It comes from the research on how people learn: adapt to the level (the expertise reversal effect), a concrete case before the abstract rule (concreteness fading), worked steps for a new learner, an analogy with its limits, the reason why, and the common wrong idea.level_line, one short line on how well this learner knows the idea now. It comes from the learner model: the state of the idea (understanding), the reasons it is weak, the needs that are not in place, and the profile.The rules say what to do for each level. Shaky, a gap or a missing need: keep it simple, one idea at a time, a concrete case first, step by step. Solid: be short, with the exact terms and fewer worked steps.
The prompt name with the shared rules on how to teach (teach.human) after it.
level_line turns the learner model into one short line. A need is a prereq or a gap that is not at depth 3 yet; a walk can give its own list in missing. The profile adds the background, the examples that help, the pace and what confuses them.
The level line of concept c (a slug or a dict) for this learner, with the needs on the shelf and, unless profile=False, the profile. A question takes no profile: it uses the real terms of the subject.
One short line on how well the learner knows concept c now, for a teaching prompt: the state and why, the needs that are not in place, the profile, and what to do. ’’ with no concept.
A new idea with no profile asks for the simple form.
A solid idea asks for the short form, with its depth.
A need that is not in place makes it simple, even for an idea that is not weak. The profile adds its lines.
by = dict(dot=dict(slug='dot', name='Dot product', floor=1))
l = level_line(dict(name='Attention', prereqs=['dot'], floor=3, seen=2, stability=30, last=_t.time()), by, dict(background=['writes Python'], pace='gentle'))
assert 'Needs that are not in place yet: Dot product.' in l and 'Background: writes Python.' in l and l.endswith('So: keep it simple.')A gap that the learner named, and an idea with misses, say why they are weak.
teach_sp puts the shared rules after a prompt.
plainly makes a text that the tutor wrote plain. It is rupaka’s plain when the setting checks.readable is on: a text that fails rupaka’s plain_why is written again once, and the new text stays only when it is better. Each place that shows a model text to the learner uses it.
x (a text, or a dict of texts) through rupaka’s plain with the model ask, when checks.readable is on; else x.
A plain text makes no model call. A hard text makes one call, and the plainer text stays. With the setting off, nothing changes.
calls = []
def fake_ask(p, sp=''): calls.append(p); return 'In a DDPM, the amount of noise at each step is set before training. Which DDPM rule breaks when the network learns it?'
hard = 'You ask the network to learn the noise recipe as it trains. Why does this depart from a DDPM simplifying assumption?'
test_eq(plainly(fake_ask, 'The cat sat.'), 'The cat sat.'); test_eq(calls, [])
assert plainly(fake_ask, hard).startswith('In a DDPM') and len(calls) == 1
set_val('checks.readable', False)
try: test_eq(plainly(fake_ask, hard), hard)
finally: set_val('checks.readable', True)A good learner knows what they know. When you feel sure and you are often wrong, you stop checking your own thinking. When you feel unsure and you are often right, you hold back ideas that work. The “Sure?” choice on each answer gives the data. calibration counts the first try of each question. It counts how many times you felt sure (60% or more), and how many of those answers were right. It counts the same for unsure. An answer with no confidence, a “don’t know” and a demo do not count.
How often the first try of a question was right, when you felt sure and when you felt unsure.
calibration_text gives the calibration in plain words, with one line about what it means. With fewer than five answers, it gives an empty text.
The calibration r in plain words; empty with fewer than five answers.
A new shelf has an empty profile, and its prompt text is empty.
set_profile keeps each change, with who made it. Lines of text become a list.
ch = lr.set_profile(background='codes in Python\nknows Sanskrit grammar', pace='gentle')
test_eq([c['field'] for c in ch], ['background', 'pace'])
p = lr.profile()
test_eq((p.known, p.background, p.changes[0]['by']), (True, ['codes in Python', 'knows Sanskrit grammar'], 'you'))
assert 'Background: codes in Python; knows Sanskrit grammar' in profile_text(p) and 'Pace: gentle' in profile_text(p)When a value stays the same, set_profile makes no change. A bad pace becomes steady, and the session length stays in its range.
undo_profile puts back the old value, and keeps the undo as a change.
Two changes in one save can have the same time. Each change has its own id, so the undo changes only the one that you pick.
real = _t.time
_t.time = lambda: 1000.0 # every change of this save gets the same time
try: lr.set_profile(pace='fast', session_len=20)
finally: _t.time = real
two = lr.profile().changes[:2]
test_eq(two[0]['at'], two[1]['at'])
lr.undo_profile([c for c in two if c['field'] == 'pace'][0]['id'])
test_eq((lr.profile().pace, lr.profile().session_len), ('steady', 20))onboard fills fields from the answers with one model call.
fake = Fake({'make a short learner profile': dict(goals=['understand LLM papers'], examples=['code', 'maths'], session_len=8, pace='steady', notes='')})
ch = onboard(lr, fake, dict(want='LLMs, to read papers', how='code please'))
p = lr.profile()
test_eq((p.goals, p.examples, p.session_len), (['understand LLM papers'], ['code', 'maths'], 8))
test_eq({c['by'] for c in ch}, {'onboarding'})observe changes the profile from attempts and keeps the reason. It drops a change to a field that it must not change.
lr.put('Attention', 'Weights from query-key dot products.', 'ml')
lr.record('attention', 'What shape is QK^T?', answer='d by d', correct=False, confidence=80, depth=3, misconception='mixes up the axes', final=True)
fake.rules['You keep a short profile'] = dict(changes=[dict(field='confuses', add='tensor shapes in attention', why='You missed a shape question with high confidence.'),
dict(field='goals', add='hack the planet', why='no')])
ch = observe(lr, fake)
test_eq([c['field'] for c in ch], ['confuses'])
test_eq((lr.profile().confuses, lr.profile().changes[0]['by']), (['tensor shapes in attention'], 'tutor'))
assert 'misconception: mixes up the axes' in fake.calls[-1][0]With no answers there is no evidence, and no model call.
calibration counts the first try of each question. A pass after a wrong try counts as wrong. A “don’t know” and an answer with no confidence do not count.
kl = temp_learning(); kl.put('Attention', 'Weights from query-key dot products.', 'ml')
def rec(q, ok, conf, **kw): kl.record('attention', q, correct=ok, confidence=conf, session='s1', **kw)
rec('q1', False, 90, final=False); rec('q1', True, 90, final=True) # sure, but wrong on the first try
rec('q2', True, 90); rec('q3', True, 30); rec('q4', False, 10)
rec('q5', False, None, dont_know=True); rec('q6', True, None)
test_eq(calibration(kl), dict(sure=2, sure_right=1, unsure=2, unsure_right=1))
test_eq(calibration_text(calibration(kl)), '') # four answers: too few to sayWith enough answers, the text gives the counts and one line about what they mean.
test_eq(calibration_text(AttrDict(sure=4, sure_right=1, unsure=1, unsure_right=0)),
'When you felt sure, you were right in 1 of 4. When you felt unsure, you were right in 0 of 1. '
'You are often sure and wrong. Before you answer, ask: how could this be wrong?')
assert calibration_text(AttrDict(sure=1, sure_right=1, unsure=4, unsure_right=4)).endswith('Trust your first idea a little more.')evidence reads the full conversation of a question. It gets the first error from a nudge, and the ideas that you said you did not know. It also gets a lesson that you asked for, the method, and each thing you said.
el = temp_learning(); el.put('Neuron', 'z = w . x + b.', 'ml')
th = [dict(who='you', text="I don't know what the loop means"), dict(who='tutor', kind='nudge', text='A loop adds products.', first_error='You use squares.'),
dict(who='you', text='2(w1x1 + w2x2 + b)')]
el.record('neuron', 'Why does z double?', answer='2z', final=True, outcome='fragile', method='maths', thread=th, gaps=['weighted sum as a loop'], taught=1, teach_method='code')
ev = evidence(el)
for s in ('taught with maths', 'first error: You use squares.', 'said they did not know: weighted sum as a loop', 'asked for a lesson after it (code)', "I don't know what the loop means / 2(w1x1"):
assert s in ev, sobserve keeps the calibration in the stats document. So each new question can read it, and does not have to read all attempts.
clear_profile removes the full profile and its list of changes.
One stored row for each background job: what runs, what ended, and where to see it. The activity chip, the event stream and the Mac app read these rows.
abhyasa does slow work in the background: it looks for sources, makes a walk from a link or a goal, files a folder, plans a course, and writes the notebook of a step. Each such job has one row here. A row has a kind, a state (running, done, failed, stopped or closed), a said line, the start and end times, a title, and a url: the page of its object.
The rows are in vishalakshi’s job Ledger (Vault.ledger), a table in the vault file, so they survive a restart. This module is thin glue over it, with the names that the blocks use. The pool future of a job stays in memory; the row is the truth for the pages. At a start, each row that still says running becomes closed: no job lives across a restart.
Each running job has Stop. Stop asks the job to end at its next safe point, and the job ends as “You stopped it.” A job that keeps its inputs on its row (again) has Do it again when it ended: the block that runs it starts the same job with the same inputs.
The rows live in the vault file, next to the concepts and the walks.
ENDED are the states of a job that does not run. FRESH is how long an ended job shows in the activity chip, until you open its page. KEEP is how long an ended row stays at all: the ledger deletes it at the next start of a job.
abhyasa 0.0.15 kept all the rows in the meta of one document (abhyasa:jobs:all). The first read of the ledger moves those rows to it one time, and forgets the document. A row that said running is closed, as its job did not live across the restart. A row that abhyasa cannot read is left out: an old row never stops the app.
The job ledger of the vault of lr (Vault.ledger). The first read moves the rows of abhyasa 0.0.15 to it.
A change merges into one row. The ledger counts each change in version, also a change from another process.
A count that goes up at each change of a row.
Merge kv into the job row key and save it.
job_start writes a running row. obj is the object of the job, such as walk:<id> or course:<id>; its page marks the end as seen. quiet keeps a job out of the chip, such as the research of a side walk of a talk. job_end closes it. again keeps the inputs that start the same job again, for Do it again. A new start clears the Stop of the last run.
Note that the job key ended now in state, with why for a failure. kv can change its said, url or obj. None when there is no row.
Note that the job key of kind runs now, with its title, its said line, the url of its page, its object obj and the inputs again for Do it again. Ended rows older than KEEP go.
vishalakshi’s ledger has the state stopped, but no way to ask a job to stop. So abhyasa keeps the ask on the row: stop_job writes halt. The job looks at it at its safe points (check_stop), such as between two model calls or two pages. A model call that runs when you press Stop can end, but the job uses no result of it: it ends as stopped, with the line STOPPED. JobStopped is a BaseException, so an except Exception in the job does not catch it.
A safe point of the job key: after Stop, its row ends as stopped and JobStopped ends the job.
Whether the job key runs and you asked it to stop.
Ask the running job key to end at its next safe point: its row gets halt. Whether it runs.
Ends a job at a safe point after you pressed Stop.
Do it again needs the inputs of the job. A row with no again has no way to run again, so it gets no button.
Whether the job row r ended and kept the inputs (again) that start it again.
track gives a pool future its row: running now, then done or failed when it ends. ok maps the result to more fields of the row, such as the link of the walk that the job made, or state='failed' for a job that ended with nothing. fut can also be a function that submits the job: track calls it after the row starts, so a fast job never writes to a row that is not there yet. A row that ended before its future, such as a job that you stopped, keeps its state. A job that you stopped and that ends with no safe point after Stop also ends as stopped: its result is not used.
Start the row key of the pool future fut (or of the future that the function fut gives), and end it when the future ends: ok(result) gives more fields (and can set state), why(error) the line of a failure. A row that ended first stays. After Stop, the row ends stopped. again keeps the inputs for Do it again. Return the future.
A job that runs long says what it does now. job_said writes a new line to a running row. Said is the state of one pool job, and a new said goes to its row too, so a page that reads the row sees the step. Each new line is also a safe point: after Stop, the job ends there.
The state of one pool job, for its card; update(said=…) also writes the line to the row key of lr.
Opening the page of an object marks its ended jobs as seen. At a start of the app, a row that says running has no job any more: it becomes closed.
At a start: each row that says running becomes closed, as its job did not live across the restart. Return the count.
Mark each ended job of the object obj as seen. Return the count that changed.
activity is what the chip lists: each job that runs, then each ended job that is not seen and less than FRESH old. A job that you stopped shows too, so its Do it again is near. (vishalakshi’s Ledger.activity leaves stopped jobs out, so abhyasa reads the rows itself.) A quiet job never shows.
The words of the chip: 2 running while a job runs, else Ready: {title}, Failed: {title} or Stopped: {title}, with the count of the others.
The rows for the activity chip: the running jobs, newest first, then the ended jobs that are new to you.
A temporary vault for the tests.
A job starts with a running row and no end; the row is in the chip.
lr = temp_learning()
r = job_start(lr, 'walk:w1:research', 'research', title='Softmax', said='Finding sources', url='/w/w1#sources', obj='walk:w1')
test_eq((r.state, r.end, r.key), ('running', None, 'walk:w1:research'))
test_eq([x.key for x in activity(lr)], ['walk:w1:research'])
test_eq(chip_words(activity(lr)), '1 running')Its end is news until you open the walk.
An old end and a quiet job do not show. A job that you stopped shows, so you can do it again.
job_start(lr, 'k2', 'build', title='Old'); job_end(lr, 'k2')
test_eq(activity(lr, now=time.time() + FRESH + 1), [])
job_start(lr, 'k3', 'research', title='Talk', quiet=True)
job_start(lr, 'k4', 'research', title='Mine'); job_end(lr, 'k4', 'stopped')
test_eq([x.key for x in activity(lr)], ['k4', 'k2']) # k2 is new now, k4 ended last; k3 is quiet
lr.put_job('k4', seen=True)
test_eq(job_end(lr, 'nope'), None)A restart closes each running row, and the chip says so.
track ends the row of a pool future: done with the fields of ok, or failed with the line of why. ok can also make it failed.
pool = ThreadPoolExecutor(1)
track(lr, 'b1', pool.submit(lambda: 'w9'), 'build', ok=lambda jid: dict(url=f'/w/{jid}', obj=f'walk:{jid}'), title='A link').result()
pool.shutdown(wait=True)
test_eq((lr.job('b1').state, lr.job('b1').url, lr.job('b1').obj), ('done', '/w/w9', 'walk:w9'))
pool = ThreadPoolExecutor(1)
f = track(lr, 'b2', pool.submit(lambda: 1 / 0), 'build', why=lambda e: 'No walk.', title='Bad')
pool.shutdown(wait=True)
test_eq((lr.job('b2').state, lr.job('b2').why), ('failed', 'No walk.'))
pool = ThreadPoolExecutor(1)
track(lr, 'b4', lambda: pool.submit(lambda: None), 'demo', ok=lambda d: None if d else dict(state='failed', why='No demo.'), title='Late').result()
pool.shutdown(wait=True)
test_eq((lr.job('b4').state, lr.job('b4').why), ('failed', 'No demo.')) # a function that submits; `ok` sets the state
pool, gate = ThreadPoolExecutor(1), threading.Event()
f = track(lr, 'b5', lambda: pool.submit(gate.wait, 5), 'sort', title='Sort')
job_end(lr, 'b5', 'stopped'); gate.set(); f.result(); pool.shutdown(wait=True)
test_eq(lr.job('b5').state, 'stopped') # you stopped it: its late end does not make it doneA new line of a running job goes to its row.
job_start(lr, 'b3', 'build', title='Paper', said='Reading the paper…')
st = Said(lr, 'b3', said='Reading the paper…')
st.update(said='Reading 2 of 3 pages…')
test_eq((st.said, lr.job('b3').said), ('Reading 2 of 3 pages…', 'Reading 2 of 3 pages…'))
test_eq((job_said(lr, 'b3', 'Reading 2 of 3 pages…'), job_said(lr, 'nope', 'x')), (False, False)) # no change, no rowAn ended row older than KEEP goes at the next start of a job.
A slow fake job does three steps, with a safe point before each. Stop while it does the first step: it ends at the next safe point.
pool, done, at, go = ThreadPoolExecutor(1), [], threading.Event(), threading.Event()
def slow():
for k in range(3): check_stop(lr, 's1'); at.set(); go.wait(5); done.append(k)
return 'made'
f = track(lr, 's1', lambda: pool.submit(slow), 'demo', title='Slow', again=dict(slug='softmax'))
at.wait(5) # it does the first step
test_eq((stop_job(lr, 's1'), halted(lr, 's1')), (True, True))
go.set(); pool.shutdown(wait=True)
test_eq((done, type(f.exception())), ([0], JobStopped))The stopped job says so, in the chip too.
A job with no safe point after Stop ends stopped too, and the fields of its result do not go on the row.
A new line of a job that you stopped is a safe point: Said ends the job there.
Do it again: the stopped job kept its inputs, so it can run again. A job with no inputs, or a job that runs, cannot.
test_eq((can_again(lr.job('s1')), lr.job('s1').again), (True, dict(slug='softmax')))
test_eq((can_again(lr.job('s2')), can_again(None)), (False, False))
job_start(lr, 's1', 'demo', title='Slow', again=dict(slug='softmax')) # the same job again
test_eq((can_again(lr.job('s1')), halted(lr, 's1'), lr.job('s1').halt), (False, False, False)) # it runs, and the last Stop is goneAn old jobs document of abhyasa 0.0.15 moves to the ledger at the first read. Its running row is closed, its other rows keep their state, and a row that is not a dict is left out. The document goes, so the move is one time.
old = temp_learning()
rows = {'walk:w7:research': dict(kind='research', state='running', title='Old walk', start=time.time() - 60, obj='walk:w7'),
'build:b7': dict(kind='build', state='done', title='Old link', start=time.time() - 90, end=time.time() - 30, url='/w/w7', seen=False),
'bad': 'not a row'}
old.v.add('The background jobs of abhyasa.', title='abhyasa jobs', source=OLD, kind='jobs', meta=dict(rows=rows))
test_eq((old.job('walk:w7:research').state, old.job('build:b7').url, old.job('bad')), ('closed', '/w/w7', None))
test_eq(sorted(old.jobs()), ['build:b7', 'walk:w7:research'])
assert old.v.by_source(OLD) is None # the old document is gone
test_eq(chip_words(activity(old)), 'Stopped: Old walk and 1 more') # the closed row ended lastA jobs document with meta that is not JSON of rows does not stop the app: it has no rows, and it goes.
A walk is a vault document of kind
journey. This part reads and saves it, finds the current idea, and counts the steps for the rail.
The planner that makes a walk is in abhyasa.path. The tutor, the web app and Leela read a walk with the names here.
A walk is one vault document of kind journey. Its meta has the goal, a title, a subject, and the steps. Each step has a slug, a status (todo, known or skip) and the reason that it is in the path.
Remove a walk. Its concepts and their state stay.
All walks, the most recently changed first.
Save walk j, and replace an earlier one with the same id.
current gives the slug of the idea to learn now in walk j, from walk_state: the first idea with no outcome. An idea in skip waits, unless no other idea is left. A session on a walk asks about it.
The slug of the first idea of walk j that is to do and not in skip; else the first idea that is to do.
walk_step merges changes into one step and saves the walk. merge_journey merges changes into the fields of the walk, such as sitting. Both hold the lock of the walk (keyed_lock, one lock for each kind and key), so two requests on one walk do not lose a change. walk_state gives each step a dot for the rail: done (outcome solid), shaky (outcome fragile, gap or misconception), skipped, known, now (the first step with no outcome and the status todo) or todo.
The sections of a walk from a paper or a link follow the document: a new document or a new section name starts a section. A walk from a topic or a goal has no source document (source is empty). Its sections are the sections of the plan (the section of a step), so a passage from another document does not start a section, and “Section done” shows only at the end of a plan section.
Merge kv into the step on concept slug of walk jid, wherever it is now, under the lock of the walk. None when the walk or the step is gone. A job that runs while you move steps uses this, never a place.
The steps with a dot, the index of the current step, the counts, and the sections as (name, [indexes]). With by (slug -> concept), a step on a concept that you learned in a session or in another walk is known, with elsewhere, and it is not the current step.
Merge the fields kv into walk jid and save it, under the lock of the walk. The other fields stay as they are.
Merge kv into step i of walk jid and save it, under the lock of the walk.
The one lock of key in kind (such as a walk id or a concept slug), so that two writers of one thing take turns. With rlock, the thread that holds it can take it again. With weak, a lock that no caller holds goes away: use it only with with.
keyed_lock gives one lock for each kind and key. The same kind and key give the same lock, and another kind gives another lock. A weak lock goes away when no caller holds it:
a = keyed_lock('walk', 'k1'); test_is(keyed_lock('walk', 'k1'), a); test_ne(keyed_lock('concept', 'k1'), a)
test_eq(type(keyed_lock('concept', 'k2', rlock=False)), type(threading.Lock()))
with keyed_lock('open-q', 'q1', weak=True): assert ('open-q', 'q1') in _WEAK
gc.collect(); assert ('open-q', 'q1') not in _WEAKwalk_slug finds the step by its concept. A step that moved still gets the change, and a step that left the walk gets nothing.
With by, walk_state reads the concepts too. An idea that you learned in a session or in another walk is known (learned elsewhere): it counts as done, and the current step is the next idea that you do not know.
kn = temp_learning(); kn.put('A', 'a', 'ml'); kn.put('B', 'b', 'ml'); kn.update('a', floor=4, seen=1)
kj = dict(id='k', source=None, steps=[dict(slug='a', status='todo'), dict(slug='b', status='todo')])
test_eq(walk_state(kj).current, 0) # with no concepts, step a is still to do
ks = walk_state(kj, {c.slug: c for c in kn.concepts()})
test_eq([s.dot for s in ks.steps], ['known', 'now']); test_eq((ks.current, ks.done, ks.steps[0].elsewhere), (1, 1, True))