sources

Add concepts to the concept graph from any text, from a vault document and from vedicreader.

This module adds concepts to the learning shelf. A model finds the core ideas in a text, and code merges them with the concepts on the shelf. The vedicreader import reads the repo files and needs no model. The tutor then asks about the concepts that this module adds.

These are the imports of the module.

Extraction from text

The model lists the core ideas of a text. Each idea has a true, self-contained statement and its prereqs. Code does the merge. When a slug or an alias matches a concept on the shelf, it is the same concept. The lists of that concept get the new items, and its learner state does not change.

EXTRACT_SP and extract_prompt ask the model for the concepts.

---

source

extract_prompt

def extract_prompt(
    text, subject:str='', known:tuple=()
):

_index finds the slug for a name or an alias. _windows cuts a long text into parts for the model.

extract adds the concepts of a text. from_vault does the same for one document of the vault.

---

source

from_vault

def from_vault(
    lr:abhyasa.store.Learning, ask, ref:str, subject:str=''
)->fastcore.foundation.L:

Extract concepts from one document of the main shelf of the vault, by doc id, source or title.

---

source

extract

def extract(
    lr:abhyasa.store.Learning, ask, text:str, subject:str='', source:dict=None
)->fastcore.foundation.L:

Add the core ideas of text to the shelf as concepts, merged with the concepts there. Return the slugs that it changed.

A fake model returns three concepts. One matches Ranking on the shelf by its alias rank.

from abhyasa.store import temp_learning
from abhyasa.llm import Fake
lr = temp_learning()
lr.put('Ranking', 'Order items by a score.', 'ml', aliases=['rank'])
fake = Fake({'Return JSON: {"concepts"': dict(concepts=[
    dict(name='Reciprocal rank fusion', statement='Sum 1/(k + rank) over lists.', aliases=['RRF'], prereqs=['rank', 'Ranked list']),
    dict(name='Ranked list', statement='Items in score order.', aliases=[], prereqs=[]),
    dict(name='Ranking', statement='Ordering by score.', aliases=['ordering'], prereqs=[])])})

extract returns the changed slugs. The prereqs become slugs, and the old concept gets the new alias and statement.

test_eq(sorted(extract(lr, fake, 'some text', 'ml', dict(src='test'))), ['ranked-list', 'ranking', 'reciprocal-rank-fusion'])
test_eq(lr.concept('reciprocal-rank-fusion').prereqs, ['ranking', 'ranked-list'])
test_eq(lr.concept('ranking').aliases, ['rank', 'ordering'])
test_eq(lr.concept('ranking').statement, 'Ordering by score.')

_windows cuts at paragraph breaks.

test_eq(len(_windows('a\n\n' * 10, size=5)), 5)

vedicreader

The import reads the files of the vedicreader repo. It does not read the website. $VEDICREADER_ROOT gives the path of the repo.

source	concepts
Sanskrit lessons	One concept for each lesson. The lesson before it in the course is its prereq. Its check pairs become ready questions at depth 4 (apply). Its recap becomes the statement.
static/grammar/terms/*.json	One concept for each term, with related links. A term with a jyotiṣa term of the same name also links to it.
vr/jyotisha/explain/*.json	One concept for each graha, rāśi and nakṣatra.

VR_ROOT is the repo path. plain removes the marks of vedicreader from a text.

---

source

plain

def plain(
    s:str
)->str:

The vedicreader text without its {…} marks and [1,2] citations.

Three generators read one source each. Each generator yields the concepts as dicts for put. _lesson_concepts reads the lessons.

_term_concepts reads the grammar terms.

_explain_concepts reads the jyotiṣa explanations. The rāśi file keys each sign by its number, so RASI gives the names.

from_vedicreader runs the generators that parts names and returns a count for each part.

---

source

from_vedicreader

def from_vedicreader(
    lr:abhyasa.store.Learning, root:NoneType=None, parts:tuple=('lessons', 'terms', 'explain')
)->dict:

Add the lessons, terms and jyotiṣa explanations of vedicreader as concepts. Keep the learner state on the shelf.

plain removes the marks and the citations.

test_eq(plain('{a} + {a} → {ā}: {nāsti}.[1,2]'), 'a + a → ā: nāsti.')

When the repo is on this machine, the lessons become a chain of concepts with prereqs and ready questions.

if VR_ROOT.exists():
    lr = temp_learning()
    n = from_vedicreader(lr, parts=('lessons',))
    assert n['lessons'] > 10
    c = lr.concept('sa-sandhi-vowels')
    assert c.prereqs and c.questions and c.subject == 'sanskrit'
    test_eq(lr.concept('sa-sounds').prereqs, [])
    test_eq(_explain_name('rasi', '0'), ('Meṣa', ['Aries', 'rāśi', 'sign']))