# profile
The tutor asks better questions when it knows the learner. A coder gets
code examples. A Sanskrit reader gets a Sanskrit example. This module
keeps one `profile` document on the learning shelf. The profile gets
data in three ways: a short first conversation, edits by the learner,
and changes that the tutor makes from evidence. The module keeps each
change with who made it and why. So the learner can see each change and
undo it. The profile stays in the vault on this machine.
Import the modules that this notebook uses.
## The fields
A field is a list of short items, or one value. `PACES` names the three
paces. `EXAMPLES` names the kinds of example a learner can like.
[`profile_text`](https://vedicreader.github.io/abhyasa/profile.html#profile_text)
gives the profile as lines for a prompt. It is empty when the profile
has no data. So the prompt for a new learner does not change.
------------------------------------------------------------------------
source
### profile_text
``` python
def profile_text(
p
)->str:
```
*The profile as prompt lines; empty when nothing is known.*
[`clean`](https://vedicreader.github.io/abhyasa/profile.html#clean)
makes a value fit its field. A list field takes a list or lines of text.
The pace must be a known pace. The session length stays between 3 and
40.
------------------------------------------------------------------------
source
### clean
``` python
def clean(
k, v
):
```
*The value `v` made to fit field `k`.*
## The document
`profile` reads the document. It uses the defaults for the fields that
are not set. `known` is False until the profile has a saved value.
------------------------------------------------------------------------
source
### Learning.profile
``` python
def profile()->fastcore.basics.AttrDict:
```
*The learner profile, with its list of changes, newest first.*
`set_profile` changes fields. It keeps each change with the old value,
the new value, who made it (`you`, `onboarding` or
[`tutor`](https://vedicreader.github.io/abhyasa/nb.html#tutor)) and why.
------------------------------------------------------------------------
source
### Learning.set_profile
``` python
def set_profile(
by:str='you', why:str='', **kv
)->list:
```
*Change fields of the profile and keep each change with who made it and
why. Return the changes.*
`undo_profile` puts back the old value of one change. It finds the
change by its time. The undo is a change too, so the list shows all the
changes.
------------------------------------------------------------------------
source
### Learning.clear_profile
``` python
def clear_profile():
```
*Remove the profile and its changes.*
------------------------------------------------------------------------
source
### Learning.undo_profile
``` python
def undo_profile(
at:float
)->list:
```
*Put back the old value of the change made at `at`.*
## The first conversation
The learner answers four questions, by voice or by text. One model call
turns the answers into fields.
------------------------------------------------------------------------
source
### onboard
``` python
def onboard(
lr:abhyasa.store.Learning, ask, answers:dict
)->list:
```
*Fill the profile from the onboarding answers. Return the changes.*
------------------------------------------------------------------------
source
### onboard_prompt
``` python
def onboard_prompt(
answers:dict
)->str:
```
*The prompt that turns the onboarding answers into profile fields.*
## The tutor learns from evidence
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
reads the recent answers: right or not, confidence, hints,
misconceptions and think-aloud. A small model call proposes at most
three changes, each with its reason.
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
applies them immediately and keeps each one with its reason. So the
learner sees them and can undo them.
------------------------------------------------------------------------
source
### observe_prompt
``` python
def observe_prompt(
p, ev:str, st:NoneType=None
)->str:
```
*The prompt that proposes changes to profile `p` from the evidence `ev`
and the method counts in the stats `st`.*
------------------------------------------------------------------------
source
### evidence
``` python
def evidence(
lr:abhyasa.store.Learning, n:int=12
)->str:
```
*The last `n` final answers as short lines for a prompt: the outcome,
the method, the first error, gaps you named, lessons you asked for, and
what you said.*
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
applies the changes. A list field takes `add` and
[`remove`](https://vedicreader.github.io/abhyasa/voice.html#remove). The
pace takes `value`. The goals and the notes are for the learner only, so
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
drops a change to them.
------------------------------------------------------------------------
source
### observe
``` python
def observe(
lr:abhyasa.store.Learning, ask, n:int=12
)->list:
```
*Change the profile from the recent answers. Return the changes, each
with its reason.*
## How well you know what you know
A good learner knows what they know. When you feel sure and you are
often wrong, you stop checking your own thinking. When you feel unsure
and you are often right, you hold back ideas that work. The “Sure?”
choice on each answer gives the data.
[`calibration`](https://vedicreader.github.io/abhyasa/profile.html#calibration)
counts the first try of each question. It counts how many times you felt
sure (60% or more), and how many of those answers were right. It counts
the same for unsure. An answer with no confidence, a “don’t know” and a
demo do not count.
------------------------------------------------------------------------
source
### calibration
``` python
def calibration(
lr:abhyasa.store.Learning, n:int=300
)->fastcore.basics.AttrDict:
```
*How often the first try of a question was right, when you felt sure and
when you felt unsure.*
[`calibration_text`](https://vedicreader.github.io/abhyasa/profile.html#calibration_text)
gives the calibration in plain words, with one line about what it means.
With fewer than five answers, it gives an empty text.
------------------------------------------------------------------------
source
### calibration_text
``` python
def calibration_text(
r
)->str:
```
*The calibration `r` in plain words; empty with fewer than five
answers.*
## Tests
A new shelf has an empty profile, and its prompt text is empty.
``` python
from abhyasa.store import temp_learning
from abhyasa.llm import Fake
lr = temp_learning()
p = lr.profile()
test_eq((p.known, p.pace, p.background, profile_text(p)), (False, 'steady', [], ''))
```
`set_profile` keeps each change, with who made it. Lines of text become
a list.
``` python
ch = lr.set_profile(background='codes in Python\nknows Sanskrit grammar', pace='gentle')
test_eq([c['field'] for c in ch], ['background', 'pace'])
p = lr.profile()
test_eq((p.known, p.background, p.changes[0]['by']), (True, ['codes in Python', 'knows Sanskrit grammar'], 'you'))
assert 'Background: codes in Python; knows Sanskrit grammar' in profile_text(p) and 'Pace: gentle' in profile_text(p)
```
When a value stays the same, `set_profile` makes no change. A bad pace
becomes `steady`, and the session length stays in its range.
``` python
test_eq(lr.set_profile(pace='gentle'), [])
test_eq((clean('pace', 'warp'), clean('session_len', '200'), clean('session_len', 'x')), ('steady', 40, 12))
```
`undo_profile` puts back the old value, and keeps the undo as a change.
``` python
at = [c for c in lr.profile().changes if c['field'] == 'pace'][0]['at']
lr.undo_profile(at)
test_eq((lr.profile().pace, lr.profile().changes[0]['why']), ('steady', 'you undid a change'))
```
[`onboard`](https://vedicreader.github.io/abhyasa/profile.html#onboard)
fills fields from the answers with one model call.
``` python
fake = Fake({'make a short learner profile': dict(goals=['understand LLM papers'], examples=['code', 'maths'], session_len=8, pace='steady', notes='')})
ch = onboard(lr, fake, dict(want='LLMs, to read papers', how='code please'))
p = lr.profile()
test_eq((p.goals, p.examples, p.session_len), (['understand LLM papers'], ['code', 'maths'], 8))
test_eq({c['by'] for c in ch}, {'onboarding'})
```
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
changes the profile from attempts and keeps the reason. It drops a
change to a field that it must not change.
``` python
lr.put('Attention', 'Weights from query-key dot products.', 'ml')
lr.record('attention', 'What shape is QK^T?', answer='d by d', correct=False, confidence=80, depth=3, misconception='mixes up the axes', final=True)
fake.rules['You keep a short profile'] = dict(changes=[dict(field='confuses', add='tensor shapes in attention', why='You missed a shape question with high confidence.'),
dict(field='goals', add='hack the planet', why='no')])
ch = observe(lr, fake)
test_eq([c['field'] for c in ch], ['confuses'])
test_eq((lr.profile().confuses, lr.profile().changes[0]['by']), (['tensor shapes in attention'], 'tutor'))
assert 'misconception: mixes up the axes' in fake.calls[-1][0]
```
With no answers there is no evidence, and no model call.
``` python
n = len(fake.calls)
test_eq(observe(temp_learning(), fake), [])
test_eq(len(fake.calls), n)
```
[`calibration`](https://vedicreader.github.io/abhyasa/profile.html#calibration)
counts the first try of each question. A pass after a wrong try counts
as wrong. A “don’t know” and an answer with no confidence do not count.
``` python
kl = temp_learning(); kl.put('Attention', 'Weights from query-key dot products.', 'ml')
def rec(q, ok, conf, **kw): kl.record('attention', q, correct=ok, confidence=conf, session='s1', **kw)
rec('q1', False, 90, final=False); rec('q1', True, 90, final=True) # sure, but wrong on the first try
rec('q2', True, 90); rec('q3', True, 30); rec('q4', False, 10)
rec('q5', False, None, dont_know=True); rec('q6', True, None)
test_eq(calibration(kl), dict(sure=2, sure_right=1, unsure=2, unsure_right=1))
test_eq(calibration_text(calibration(kl)), '') # four answers: too few to say
```
With enough answers, the text gives the counts and one line about what
they mean.
``` python
test_eq(calibration_text(AttrDict(sure=4, sure_right=1, unsure=1, unsure_right=0)),
'When you felt sure, you were right in 1 of 4. When you felt unsure, you were right in 0 of 1. '
'You are often sure and wrong. Before you answer, ask: how could this be wrong?')
assert calibration_text(AttrDict(sure=1, sure_right=1, unsure=4, unsure_right=4)).endswith('Trust your first idea a little more.')
```
[`evidence`](https://vedicreader.github.io/abhyasa/profile.html#evidence)
reads the full conversation of a question. It gets the first error from
a nudge, and the ideas that you said you did not know. It also gets a
lesson that you asked for, the method, and each thing you said.
``` python
el = temp_learning(); el.put('Neuron', 'z = w . x + b.', 'ml')
th = [dict(who='you', text="I don't know what the loop means"), dict(who='tutor', kind='nudge', text='A loop adds products.', first_error='You use squares.'),
dict(who='you', text='2(w1x1 + w2x2 + b)')]
el.record('neuron', 'Why does z double?', answer='2z', final=True, outcome='fragile', method='maths', thread=th, gaps=['weighted sum as a loop'], taught=1, teach_method='code')
ev = evidence(el)
for s in ('taught with maths', 'first error: You use squares.', 'said they did not know: weighted sum as a loop', 'asked for a lesson after it (code)', "I don't know what the loop means / 2(w1x1"):
assert s in ev, s
```
[`observe`](https://vedicreader.github.io/abhyasa/profile.html#observe)
keeps the calibration in the stats document. So each new question can
read it, and does not have to read all attempts.
``` python
observe(el, Fake({'keep a short profile': dict(changes=[])}))
test_eq(set(el.stats().calib), {'sure', 'sure_right', 'unsure', 'unsure_right'})
test_eq((clean('explore', 'often'), clean('explore', 'always')), ('often', 'sometimes'))
```
`clear_profile` removes the full profile and its list of changes.
``` python
lr.clear_profile()
test_eq((lr.profile().known, lr.profile().changes), (False, []))
```