# sources


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

This module adds concepts to the `learning` shelf. A model finds the
core ideas in a text, and code merges them with the concepts on the
shelf. The vedicreader import reads the repo files and needs no model.
The tutor then asks about the concepts that this module adds.

These are the imports of the module.

## Extraction from text

The model lists the core ideas of a text. Each idea has a true,
self-contained statement and its prereqs. Code does the merge. When a
slug or an alias matches a concept on the shelf, it is the same concept.
The lists of that concept get the new items, and its learner state does
not change.

`EXTRACT_SP` and
[`extract_prompt`](https://vedicreader.github.io/abhyasa/sources.html#extract_prompt)
ask the model for the concepts.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/sources.py#L24"
target="_blank" style="float:right; font-size:smaller">source</a>

### extract_prompt

``` python
def extract_prompt(
    text, subject:str='', known:tuple=()
):
```

[`_index`](https://vedicreader.github.io/abhyasa/sources.html#_index)
finds the slug for a name or an alias.
[`_windows`](https://vedicreader.github.io/abhyasa/sources.html#_windows)
cuts a long text into parts for the model.

[`extract`](https://vedicreader.github.io/abhyasa/sources.html#extract)
adds the concepts of a text.
[`from_vault`](https://vedicreader.github.io/abhyasa/sources.html#from_vault)
does the same for one document of the vault.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/sources.py#L66"
target="_blank" style="float:right; font-size:smaller">source</a>

### from_vault

``` python
def from_vault(
    lr:abhyasa.store.Learning, ask, ref:str, subject:str=''
)->fastcore.foundation.L:
```

*Extract concepts from one document of the main shelf of the vault, by
doc id, source or title.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/sources.py#L47"
target="_blank" style="float:right; font-size:smaller">source</a>

### extract

``` python
def extract(
    lr:abhyasa.store.Learning, ask, text:str, subject:str='', source:dict=None
)->fastcore.foundation.L:
```

*Add the core ideas of `text` to the shelf as concepts, merged with the
concepts there. Return the slugs that it changed.*

A fake model returns three concepts. One matches `Ranking` on the shelf
by its alias `rank`.

``` python
from abhyasa.store import temp_learning
from abhyasa.llm import Fake
lr = temp_learning()
lr.put('Ranking', 'Order items by a score.', 'ml', aliases=['rank'])
fake = Fake({'Return JSON: {"concepts"': dict(concepts=[
    dict(name='Reciprocal rank fusion', statement='Sum 1/(k + rank) over lists.', aliases=['RRF'], prereqs=['rank', 'Ranked list']),
    dict(name='Ranked list', statement='Items in score order.', aliases=[], prereqs=[]),
    dict(name='Ranking', statement='Ordering by score.', aliases=['ordering'], prereqs=[])])})
```

[`extract`](https://vedicreader.github.io/abhyasa/sources.html#extract)
returns the changed slugs. The prereqs become slugs, and the old concept
gets the new alias and statement.

``` python
test_eq(sorted(extract(lr, fake, 'some text', 'ml', dict(src='test'))), ['ranked-list', 'ranking', 'reciprocal-rank-fusion'])
test_eq(lr.concept('reciprocal-rank-fusion').prereqs, ['ranking', 'ranked-list'])
test_eq(lr.concept('ranking').aliases, ['rank', 'ordering'])
test_eq(lr.concept('ranking').statement, 'Ordering by score.')
```

[`_windows`](https://vedicreader.github.io/abhyasa/sources.html#_windows)
cuts at paragraph breaks.

``` python
test_eq(len(_windows('a\n\n' * 10, size=5)), 5)
```

## vedicreader

The import reads the files of the vedicreader repo. It does not read the
website. `$VEDICREADER_ROOT` gives the path of the repo.

<table>
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr>
<th>source</th>
<th>concepts</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sanskrit lessons</td>
<td>One concept for each lesson. The lesson before it in the course is
its prereq. Its <code>check</code> pairs become ready questions at depth
4 (apply). Its recap becomes the statement.</td>
</tr>
<tr>
<td><code>static/grammar/terms/*.json</code></td>
<td>One concept for each term, with <a
href="https://vedicreader.github.io/abhyasa/journey.html#related"><code>related</code></a>
links. A term with a jyotiṣa term of the same name also links to
it.</td>
</tr>
<tr>
<td><code>vr/jyotisha/explain/*.json</code></td>
<td>One concept for each graha, rāśi and nakṣatra.</td>
</tr>
</tbody>
</table>

`VR_ROOT` is the repo path.
[`plain`](https://vedicreader.github.io/abhyasa/sources.html#plain)
removes the marks of vedicreader from a text.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/sources.py#L74"
target="_blank" style="float:right; font-size:smaller">source</a>

### plain

``` python
def plain(
    s:str
)->str:
```

*The vedicreader text without its `{…}` marks and `[1,2]` citations.*

Three generators read one source each. Each generator yields the
concepts as dicts for `put`.
[`_lesson_concepts`](https://vedicreader.github.io/abhyasa/sources.html#_lesson_concepts)
reads the lessons.

[`_term_concepts`](https://vedicreader.github.io/abhyasa/sources.html#_term_concepts)
reads the grammar terms.

[`_explain_concepts`](https://vedicreader.github.io/abhyasa/sources.html#_explain_concepts)
reads the jyotiṣa explanations. The rāśi file keys each sign by its
number, so `RASI` gives the names.

[`from_vedicreader`](https://vedicreader.github.io/abhyasa/sources.html#from_vedicreader)
runs the generators that `parts` names and returns a count for each
part.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/abhyasa/blob/main/abhyasa/sources.py#L127"
target="_blank" style="float:right; font-size:smaller">source</a>

### from_vedicreader

``` python
def from_vedicreader(
    lr:abhyasa.store.Learning, root:NoneType=None, parts:tuple=('lessons', 'terms', 'explain')
)->dict:
```

*Add the lessons, terms and jyotiṣa explanations of vedicreader as
concepts. Keep the learner state on the shelf.*

[`plain`](https://vedicreader.github.io/abhyasa/sources.html#plain)
removes the marks and the citations.

``` python
test_eq(plain('{a} + {a} → {ā}: {nāsti}.[1,2]'), 'a + a → ā: nāsti.')
```

When the repo is on this machine, the lessons become a chain of concepts
with prereqs and ready questions.

``` python
if VR_ROOT.exists():
    lr = temp_learning()
    n = from_vedicreader(lr, parts=('lessons',))
    assert n['lessons'] > 10
    c = lr.concept('sa-sandhi-vowels')
    assert c.prereqs and c.questions and c.subject == 'sanskrit'
    test_eq(lr.concept('sa-sounds').prereqs, [])
    test_eq(_explain_name('rasi', '0'), ('Meṣa', ['Aries', 'rāśi', 'sign']))
```