# nb
This module brings the tutor into a notebook.
[`learn`](https://vedicreader.github.io/abhyasa/nb.html#learn) shows a
question card in the output of a cell.
[`exercise`](https://vedicreader.github.io/abhyasa/nb.html#exercise)
gives a code exercise, and its tests run on your function in the kernel.
Both use the same
[`Tutor`](https://vedicreader.github.io/abhyasa/tutor.html#tutor) and
the same `learning` shelf as the web app.
These are the imports of the module.
## Use
Leela shows ipywidgets, so the card is a widget in the output of the
cell. Leela keeps cell metadata that it does not know. Thus the cells
from
[`write_exercise`](https://vedicreader.github.io/abhyasa/nb.html#write_exercise)
keep their `metadata.abhyasa`.
``` python
from abhyasa.nb import learn, exercise
learn('reciprocal-rank-fusion') # a question card
ex = exercise('reciprocal-rank-fusion')
def rrf(lists, k=60): ... # your code, in the next cell
ex.check(rrf) # runs the tests, records the attempt
```
## The tutor
[`tutor`](https://vedicreader.github.io/abhyasa/nb.html#tutor) makes the
tutor of the notebook one time. A call with a store or a model makes a
new one.
------------------------------------------------------------------------
source
### tutor
``` python
def tutor(
learning:NoneType=None, ask:NoneType=None
)->abhyasa.tutor.Tutor:
```
*The tutor of the notebook, made one time: the default vault and
[`rishi_ask`](https://vedicreader.github.io/abhyasa/llm.html#rishi_ask),
unless you give others.*
## The question card
[`Card`](https://vedicreader.github.io/abhyasa/nb.html#card) has a box
for the answer, a box for the think-aloud, a confidence slider and four
buttons.
------------------------------------------------------------------------
source
### Card
``` python
def Card(
sess:abhyasa.tutor.Session
):
```
*A question card widget over a
[`Session`](https://vedicreader.github.io/abhyasa/tutor.html#session):
answer, think aloud, confidence, hint, and next.*
[`fb_html`](https://vedicreader.github.io/abhyasa/nb.html#fb_html) shows
a result in the card.
[`learn`](https://vedicreader.github.io/abhyasa/nb.html#learn) starts a
session in a card.
------------------------------------------------------------------------
source
### learn
``` python
def learn(
focus:str=None, subject:str=None, limit:int=12
)->__main__.Card:
```
*A question card for a session, on one concept when you name it.*
------------------------------------------------------------------------
source
### fb_html
``` python
def fb_html(
r
)->str:
```
*A result as HTML.*
After a miss on a concept that has a demo, the card shows the demo under
the feedback.
[`demo_view`](https://vedicreader.github.io/abhyasa/nb.html#demo_view)
gives the demo as a notebook output. It uses `show` from rupaka, and
leela draws the output in a sandboxed frame.
------------------------------------------------------------------------
source
### demo_view
``` python
def demo_view(
r, learning
):
```
*The demo of result `r` as a notebook output, or None.*
A result with a demo gives an output with the rupaka mime and a full
HTML page. A result without one gives nothing.
``` python
from abhyasa.store import temp_learning
from rupaka.spec import gallery
from rupaka.render import MIME
dl = temp_learning(); dl.put('Vowel sandhi', 'Vowels join by rule.', 'sanskrit')
dl.put_demo(gallery()['vowel-sandhi'], 'vowel-sandhi')
mb = demo_view(AttrDict(demo='vowel-sandhi'), dl)._repr_mimebundle_()
assert MIME in mb and 'source
### code_prompt
``` python
def code_prompt(
c
):
```
The model writes the tests, so the notebook does not trust them.
[`safe_test`](https://vedicreader.github.io/abhyasa/nb.html#safe_test)
accepts only one `assert`. The assert can use only the function of the
learner, literals, comparisons, arithmetic and a short list of safe
builtins. It cannot use an import, a name or an attribute with a leading
underscore, `open`, `exec` or `eval`.
[`validate_exercise`](https://vedicreader.github.io/abhyasa/nb.html#validate_exercise)
checks the shape of the reply of the model. A test runs with only the
safe builtins. When a test runs on the main thread (as in a kernel), it
also has a time limit. Thus an endless loop fails the test.
------------------------------------------------------------------------
source
### time_limit
``` python
def time_limit(
secs:float=2.0
):
```
*Raise `TimeoutError` after `secs`, on the main thread; elsewhere no
limit is possible, so none is set.*
------------------------------------------------------------------------
source
### validate_exercise
``` python
def validate_exercise(
r
)->fastcore.basics.AttrDict:
```
*The model’s exercise, checked: task, fname, stub and 1-12 safe tests.
Raise `ValueError` with the reason.*
------------------------------------------------------------------------
source
### safe_test
``` python
def safe_test(
t:str, fname:str
)->bool:
```
*Whether `t` is one plain `assert` that uses only `fname`, literals and
the safe builtins.*
[`Exercise`](https://vedicreader.github.io/abhyasa/nb.html#exercise)
keeps one exercise open on the tutor. `check` grades it with the same
tries and lessons as a question.
------------------------------------------------------------------------
source
### Exercise
``` python
def Exercise(
tut:abhyasa.tutor.Tutor, slug:str
):
```
*One code exercise on a concept. `check(fn)` runs its tests on your
function.*
[`fb_text`](https://vedicreader.github.io/abhyasa/nb.html#fb_text) shows
a result as text.
[`exercise`](https://vedicreader.github.io/abhyasa/nb.html#exercise)
makes an exercise on the tutor of the notebook.
------------------------------------------------------------------------
source
### exercise
``` python
def exercise(
slug:str
)->__main__.Exercise:
```
*A code exercise on concept `slug`.*
------------------------------------------------------------------------
source
### fb_text
``` python
def fb_text(
r
):
```
## Exercise notebooks
[`write_exercise`](https://vedicreader.github.io/abhyasa/nb.html#write_exercise)
writes the exercise into tagged cells of a new notebook file, for you to
open later. It never writes to a notebook that is already there.
------------------------------------------------------------------------
source
### write_exercise
``` python
def write_exercise(
ex:__main__.Exercise, path
)->pathlib.Path:
```
*A new notebook at `path` with the task, a stub cell and the check. Do
not write over a file.*
## Tests
The first tests check the safety rules. Then a fake model writes a full
exercise.
[`safe_test`](https://vedicreader.github.io/abhyasa/nb.html#safe_test)
refuses an import, a name with two leading underscores, a call to
anything but the function and the safe builtins, and code that is not
one assert.
``` python
assert safe_test("assert rrf([['a'], ['b']], k=0) == ['a', 'b']", 'rrf')
assert safe_test("assert all(x in 'ab' for x in rrf([['a', 'b']]))", 'rrf')
for bad in ("import os", "assert __import__('os')", "assert rrf.__globals__", "assert open('/etc/passwd')", "assert exec('1')",
"rrf([])", "while True: pass", "assert rrf(1); import os", "assert (lambda: 1)()"):
assert not safe_test(bad, 'rrf'), bad
```
[`validate_exercise`](https://vedicreader.github.io/abhyasa/nb.html#validate_exercise)
refuses an empty or a text `tests`, a missing name, and an unsafe test,
with a reason.
``` python
good = dict(task='t', fname='f', stub='def f(): ...', tests=['assert f() == 1'])
test_eq(validate_exercise(good).fname, 'f')
for bad, why in ((dict(good, tests=[]), 'list of 1 to 12'), (dict(good, tests='assert f()'), 'list of 1 to 12'),
({k: v for k, v in good.items() if k != 'fname'}, 'fname'), (dict(good, tests=['import os']), 'not one plain assert'), ([], 'JSON object')):
test_fail(lambda: validate_exercise(bad), contains=why)
```
A test that runs too long fails after the time limit, and the kernel
continues. Off the main thread, a limit is not possible, so the code
sets none.
``` python
import threading, time
if threading.current_thread() is threading.main_thread(): # a kernel runs cells here, so the limit works
t0 = time.time()
try:
with time_limit(.2):
for _ in range(10**10): pass # long, but never endless, so no test runner can hang on it
raise AssertionError('no timeout')
except TimeoutError: assert time.time() - t0 < 2
else: # nbdev-test runs cells on another thread: no signal is possible there, so the limit steps aside
with time_limit(.2): pass
```
A fake model writes an exercise with two tests. The tutor of the
notebook uses a temporary shelf.
``` python
from abhyasa.store import temp_learning
from abhyasa.llm import Fake
lr = temp_learning()
lr.put('Reciprocal rank fusion', 'Merge ranked lists by summing 1/(k + rank).', 'ml')
fake = Fake({'Return JSON: {"task"': dict(task='Write rrf(lists, k) that returns items by fused score.', fname='rrf',
stub='def rrf(lists, k=60):\n ...', tests=["assert rrf([['a','b'],['b','a']])[0] in 'ab'", "assert rrf([['a'],['b']], k=0) == ['a','b']"],
solution='...'), 'Teach the smallest step': 'Sum 1/(k+r). Check: what if k is large?'},
dflt=dict(question='Why ranks?', answer='Comparable.', rubric=['comparable']))
t = tutor(lr, fake)
ex = exercise('reciprocal-rank-fusion')
```
A wrong function fails both tests, and the exercise stays open.
``` python
def rrf(lists, k=60): return []
assert ex.check(rrf).startswith('0 of 2 tests pass')
```
A correct function passes after the failed try, so the outcome is
`fragile` and the concept is `shaky` at depth 4.
``` python
def rrf(lists, k=60):
s = {}
for l in lists:
for r, x in enumerate(l): s[x] = s.get(x, 0) + 1 / (k + r + 1)
return sorted(s, key=lambda x: (-s[x], x))
out = ex.check(rrf)
assert out.startswith('fragile'), out # passed after a failed try
test_eq(lr.concept('reciprocal-rank-fusion').shaky, 4)
```
[`write_exercise`](https://vedicreader.github.io/abhyasa/nb.html#write_exercise)
tags the cells and does not write over a file.
``` python
import tempfile
from fastcore.nbio import read_nb
ex2 = exercise('reciprocal-rank-fusion')
p = write_exercise(ex2, Path(tempfile.mkdtemp())/'rrf.ipynb')
nb = read_nb(p)
test_eq(nb.cells[1].metadata['abhyasa']['concept'], 'reciprocal-rank-fusion')
test_fail(lambda: write_exercise(ex2, p), exc=FileExistsError)
```
[`learn`](https://vedicreader.github.io/abhyasa/nb.html#learn) shows a
card with the first question.
``` python
c = learn('reciprocal-rank-fusion')
assert 'Why ranks?' in c.q.value
```