# board


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

A learner looks at a page and says “I don’t get alpha bar”. A tutor
answers on the page, as a teacher does at a whiteboard. It circles the
term, draws an arrow to a label in the figure, moves the slider of the
demo, fills a small worked example in a side panel, and speaks one short
line for each step.

This module is the part of the board that knows nothing about learning:

- the **board language**: a closed list of drawing commands, each tied
  to a thing on the page (an *anchor*), and the check of each line of
  the model’s reply;
- the **context**: what the page shows, cleaned and cut to size;
- the **video cues**: a quote from the transcript of a video, at its
  time;
- the **stream**: each step goes to the page as soon as it is whole.

`static/board.js` draws a checked board on the page, and
`static/texparts.js` gives a formula the ids of its parts. abhyasa and
Leela both use this module and these files. The decisions about learning
stay in abhyasa: the tutor tiers (tutor, then vault, then web), the
trust rules of sources, and the prompts.

Import the helpers. The module needs only fastcore.

## The files

A host serves the files of the board.
[`assets`](https://vedicreader.github.io/rupaka/board.html#assets) gives
their paths. `bundle('js')` is `texparts.js` and `board.js` as one
script, and `bundle('css')` is the paper tokens and `board.css` as one
style sheet. The pen libraries (Rough.js and perfect-freehand, MIT) are
in the folder `libs`. The board loads them on its first use from the URL
folder `host.assets` (default `/vendor/`), so a host serves `libs`
there. The board’s side panel uses the hand font Patrick Hand: a host
loads it with
[`rupaka.render.font_css`](https://vedicreader.github.io/rupaka/render.html#font_css).

The script sets `window.rupakaBoard` and `window.rupakaTex`. It sets the
old names `window.abhyasaBoard` and `window.abhyasaTex` to the same
objects, so a host that uses the old names works.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L30"
target="_blank" style="float:right; font-size:smaller">source</a>

### bundle

``` python
def bundle(
    kind:str='js'
)->str:
```

*The board as one script (`js`) or one style sheet (`css`).*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L25"
target="_blank" style="float:right; font-size:smaller">source</a>

### assets

``` python
def assets()->fastcore.basics.AttrDict:
```

*The files of the board: `js` and `css` in bundle order, `tex` (the
formula parts), and `libs`, the folder of the pen libraries and their
licences.*

Each file is in the package, and the script names the new global and the
old one:

``` python
a = assets()
assert all(p.exists() for p in [*a.js, *a.css, a.tex]) and all((a.libs/f).exists() for f in (*LIBS, 'LICENSES.md'))
js = bundle()
assert 'window.rupakaBoard = window.abhyasaBoard' in js and 'window.rupakaTex = window.abhyasaTex' in js
assert '--rk-paper:' in bundle('css') and '.board-panel' in bundle('css')
```

### The board language

A reply is JSON lines: one object on each line. The server checks each
line alone, so a bad line drops and the rest draws. There are 17
commands. The model never writes HTML, CSS, script or a URL.

<table>
<colgroup>
<col style="width: 33%" />
<col style="width: 33%" />
<col style="width: 33%" />
</colgroup>
<thead>
<tr>
<th>op</th>
<th>fields</th>
<th>what it does</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>step</code></td>
<td><code>say</code>, and <code>src</code> with <code>quote</code></td>
<td>starts a step; <code>say</code> is the spoken line. A quote is word
for word from source <code>src</code></td>
</tr>
<tr>
<td><code>circle</code>, <code>box</code>, <code>underline</code></td>
<td><code>at</code></td>
<td>a hand-drawn ring, box or line on the anchor</td>
</tr>
<tr>
<td><code>arrow</code></td>
<td><code>from</code>, <code>to</code></td>
<td>a curved arrow between two anchors</td>
</tr>
<tr>
<td><code>label</code></td>
<td><code>at</code>, <code>text</code>, <code>side</code></td>
<td>a short note beside the anchor</td>
</tr>
<tr>
<td><code>sketch</code></td>
<td><code>at</code>, <code>pts</code></td>
<td>a free line in the box of a figure anchor, to mark a part of the
figure</td>
</tr>
<tr>
<td><a
href="https://vedicreader.github.io/rupaka/spec.html#point"><code>point</code></a></td>
<td><code>at</code></td>
<td>moves a pointer mark to the anchor</td>
</tr>
<tr>
<td><code>clear</code></td>
<td><code>n</code> (optional)</td>
<td>removes the marks of all steps, or of step <code>n</code></td>
</tr>
<tr>
<td><code>demo_set</code></td>
<td><code>set</code></td>
<td>changes inputs of the step’s rupaka demo, only after its gate is
open</td>
</tr>
<tr>
<td><code>video_at</code></td>
<td><code>t</code>, <code>quote</code></td>
<td>moves the explainer video to <code>t</code> seconds, and quotes its
transcript there</td>
</tr>
<tr>
<td><code>panel</code></td>
<td><code>title</code>, <code>head</code>, <code>cols</code>,
<code>numbered</code>, <code>stages</code> (each optional)</td>
<td>starts the side panel at the top right of the window: a small title,
a bold heading and the names of two columns. With <code>stages</code>,
its three rows are a case, a picture and the symbols. A new
<code>panel</code> starts a new beat</td>
</tr>
<tr>
<td><a
href="https://vedicreader.github.io/rupaka/spec.html#row"><code>row</code></a></td>
<td><code>cells</code>, and <code>to</code>, <code>at</code>,
<code>chain</code> (optional)</td>
<td>adds one row to the side panel: the cells as boxes, then an arrow
and <code>to</code>. With <code>at</code>, the page circles that thing
and a hand-drawn curve links it to the row. With <code>chain</code>, the
cells are words with arrows between them. A cell
<code>{"blank": value}</code> hides its value until the next step</td>
</tr>
<tr>
<td><code>note</code></td>
<td><code>text</code></td>
<td>one quiet line in the side panel</td>
</tr>
<tr>
<td><code>node</code></td>
<td><code>id</code>, <code>text</code></td>
<td>one box of a flow diagram in the side panel. <code>id</code> is a
short name (a letter, then letters, digits or <code>_</code>)</td>
</tr>
<tr>
<td><code>edge</code></td>
<td><code>from</code>, <code>to</code>, <code>text</code>
(optional)</td>
<td>an arrow from box <code>from</code> to box <code>to</code> of the
flow diagram, with a short label. An arrow back to an earlier box is a
loop</td>
</tr>
<tr>
<td><code>myth</code></td>
<td><code>think</code>, <code>fact</code>, <code>why</code></td>
<td>a card of a wrong idea: “You might think” <code>think</code>, “In
fact” <code>fact</code>, and one line of why</td>
</tr>
</tbody>
</table>

Each stroke can have `w`: the word of `say` where it starts.

A `sketch` marks a part of a figure only, so no shape covers the words
of the page. A picture of an idea goes in the side panel. For a formula,
the tutor gives an example from day-to-day life and marks the page’s own
formula.

The side panel is the place for a small concrete example (**Explain**).
The tutor builds it one row in each step, from the real values on the
page: its row names the anchor of the value in `at`. The page circles
the value and draws a curve from it to the row, so the model writes no
circle for it. A process of 3 to 5 steps is a panel with `numbered`
rows, or one `chain` row. A pipeline, a loop or a branch is a flow
diagram: boxes (`node`) and arrows between them (`edge`), one box or two
in each step, so the diagram grows with the talk.

Three more shapes come from the research on teaching. A *faded worked
example* solves its first row, and each later row hides one more part as
a blank, for the learner to fill: the blank shows its value when the
next step plays, or when the learner clicks it. A *wrong idea* card
(`myth`) puts a common wrong idea next to the right one, with one line
of why. A panel with `stages` goes from the concrete to the abstract in
three rows: a case, a picture (a chain), then the symbols. A new `panel`
line starts the next beat, for example from the values on the page to
the big idea. The side panel is the one picture of Explain. Show is a
rupaka demo.

The first line can be a header:
`{"op":"v","n":1,"sure":0.9,"need":"none"}`. `sure` is how sure the
model is, from 0 to 1. `need` says what it needs to be sure: `none`,
`vault` or `web`.

### Anchors

A stroke names a thing on the page. The page sends a list of anchors,
each with an `id`, a `kind` and its `text`. A formula also has its
`tokens` (the TeX of its parts), a figure has its `labels` (a box for
each label in the image), and the demo has its inputs.

<table>
<colgroup>
<col style="width: 33%" />
<col style="width: 33%" />
<col style="width: 33%" />
</colgroup>
<thead>
<tr>
<th>form</th>
<th>example</th>
<th>the place</th>
</tr>
</thead>
<tbody>
<tr>
<td>an id</td>
<td><code>"passage"</code></td>
<td>the whole anchor</td>
</tr>
<tr>
<td>words</td>
<td><code>{"a": "passage", "text": "closed form"}</code></td>
<td>the words in the anchor</td>
</tr>
<tr>
<td>a maths token</td>
<td><code>{"a": "m1", "tex": "\\bar\\alpha_t"}</code></td>
<td>the part of the formula</td>
</tr>
<tr>
<td>a figure label</td>
<td><code>{"a": "figure", "label": "a"}</code></td>
<td>the label in the image</td>
</tr>
<tr>
<td>a box</td>
<td><code>{"a": "figure", "box": [100, 200, 400, 600]}</code></td>
<td>a part of the anchor, from 0 to 1000</td>
</tr>
<tr>
<td>a point</td>
<td><code>{"a": "m1", "pt": [1100, 500]}</code></td>
<td>a point beside the anchor, for the far end of an arrow</td>
</tr>
</tbody>
</table>

Two ways to write the same TeX must match.
[`norm_tex`](https://vedicreader.github.io/rupaka/board.html#norm_tex)
takes out the spaces and the braces around one symbol.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L60"
target="_blank" style="float:right; font-size:smaller">source</a>

### norm_tex

``` python
def norm_tex(
    s:str
)->str:
```

*`s` with no white space, and with no braces around one character or one
command, so `\bar{\alpha}_{t}` is `\bar\alpha_t`.*

Braces and spaces do not change the match:

``` python
test_eq(norm_tex(r'\bar{\alpha}_{t}'), norm_tex(r'\bar\alpha_t'))
test_eq(norm_tex(r'\sqrt{\bar\alpha_t}'), r'\sqrt{\bar\alpha_t}')   # a group of two symbols keeps its braces
test_ne(norm_tex(r'\alpha_t'), norm_tex(r'\bar\alpha_t'))
```

Words match with no case and with white space made single, as the page
shows them.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L67"
target="_blank" style="float:right; font-size:smaller">source</a>

### flat

``` python
def flat(
    s
)->str:
```

[`anchor_why`](https://vedicreader.github.io/rupaka/board.html#anchor_why)
gives the reason that an anchor is not on the page, or `''` when it is.
The context `ctx` has the anchors by id in `ctx.by`.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L96"
target="_blank" style="float:right; font-size:smaller">source</a>

### anchor_why

``` python
def anchor_why(
    at, ctx
)->str:
```

*Why `at` names no anchor of `ctx`, or ’’ when it does.*

A small context for the tests: a passage, the DDPM formula with its
tokens, a triangle figure with labels `a` and `b`, and a demo with the
input `t`.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L112"
target="_blank" style="float:right; font-size:smaller">source</a>

### by_id

``` python
def by_id(
    ctx
)->fastcore.basics.AttrDict:
```

*`ctx` with `by`: its anchors by id.*

``` python
DDPM = r'q(\mathbf{x}_t\mid\mathbf{x}_0)=\mathcal{N}(\mathbf{x}_t;\sqrt{\bar\alpha_t}\mathbf{x}_0,(1-\bar\alpha_t)\mathbf{I})'
ctx = by_id(dict(anchors=[dict(id='passage', kind='passage', text='It admits sampling x_t at an arbitrary timestep t in closed form.'),
                          dict(id='m1', kind='formula', text=DDPM, tokens=[r'\bar\alpha_t', r'\sqrt{\bar\alpha_t}', r'\mathbf{x}_0', r'(1-\bar\alpha_t)']),
                          dict(id='out', kind='text', text='tensor([18]) gives 47. 1.0 and 1.0 give 1.414, and 2.0 gives 2.0. Guess, compare, then update the weights.'),
                          dict(id='figure', kind='figure', text='A right triangle', labels=dict(a=[100, 420, 160, 500], b=[480, 880, 540, 960]))],
                 demo=dict(inputs=dict(t=dict(value=500, min=0, max=1000, label='Step t')), open=False)))
```

Each form finds its anchor, and a wrong name does not:

``` python
test_eq(anchor_why('passage', ctx), '')
test_eq(anchor_why(dict(a='m1', tex=r'\bar{\alpha}_t'), ctx), '')
test_eq(anchor_why(dict(a='passage', text='Closed  form'), ctx), '')
test_eq(anchor_why(dict(a='figure', label='a'), ctx), '')
test_eq(anchor_why(dict(a='figure', box=[0, 0, 500, 500]), ctx), '')
assert anchor_why('e7', ctx)                                  # an id that the page did not send
assert anchor_why(dict(a='m1', tex=r'\beta_t'), ctx)          # a token that is not in the formula
assert anchor_why(dict(a='passage', text='open form'), ctx)   # words that are not in the passage
assert anchor_why(dict(a='figure', label='c'), ctx)           # a label that is not in the figure
assert anchor_why(dict(a='figure', box=[500, 0, 100, 10]), ctx)
assert anchor_why(dict(a='m1', tex=r'\bar\alpha_t', text='x'), ctx)   # two forms at once
```

Words mark a thing only when the page shows them in one place. A part of
a word does not count, so “form” does not mark the “form” of
“transform”. Words that are in the anchor two times drop, because the
page cannot know which place the model means. Words inside a formula
drop too: the page draws the formula as maths, so a mark goes on its
part with `tex`.

``` python
tctx = by_id(dict(anchors=[dict(id='p', kind='passage', text='A transform maps symbol representations $(x_1, \\dots, x_n)$ to continuous representations $z$.')]))
test_eq(anchor_why(dict(a='p', text='continuous representations'), tctx), '')
test_eq(anchor_why(dict(a='p', text='form'), tctx), "no words 'form' in p")
test_eq(anchor_why(dict(a='p', text='representations'), tctx), "the words 'representations' are in p more than once")
test_eq(anchor_why(dict(a='p', text='x_1'), tctx), 'words inside a formula of p: mark its part with tex')
test_eq(anchor_why(dict(a='p', text='$z$'), tctx), 'words inside a formula of p: mark its part with tex')
```

### The check of one line

[`check_line`](https://vedicreader.github.io/rupaka/board.html#check_line)
gives the reason that a line drops, or `''`. Each string in a line is
checked too: no tag, no script and no link can get in, also as the text
of a label.

A cue of the explainer video is `[t, text]`.
[`cue_at`](https://vedicreader.github.io/rupaka/board.html#cue_at) gives
the cue that plays at `t` seconds.
[`cue_quote`](https://vedicreader.github.io/rupaka/board.html#cue_quote)
finds a quote in the cue at `t` or in the cue before or after it. It
gives the start of the cue where the quote begins, and the quote as the
transcript says it. The board seeks the video to that start, so a time
that the model gives a little late still lands on the right cue.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L134"
target="_blank" style="float:right; font-size:smaller">source</a>

### cue_quote

``` python
def cue_quote(
    cues, t:float, quote:str
):
```

*`(start, words)` when `quote` (3 words or more) is word for word in the
cue at `t` with the cue before and after it: the start of the cue where
it begins, and the quote as the transcript says it. Else None.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L130"
target="_blank" style="float:right; font-size:smaller">source</a>

### cue_at

``` python
def cue_at(
    cues, t:float
):
```

*The cue `(start, text)` that plays at `t`: the last one that starts at
or before `t`. None when there is none.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L212"
target="_blank" style="float:right; font-size:smaller">source</a>

### check_line

``` python
def check_line(
    o, ctx
)->str:
```

*Why the reply line `o` drops, or ’’ when it is good. `ctx` comes from
[`by_id`](https://vedicreader.github.io/rupaka/board.html#by_id).*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L166"
target="_blank" style="float:right; font-size:smaller">source</a>

### plain_say

``` python
def plain_say(
    say:str
)->str:
```

*The spoken line `say` with no semicolon: each part is a sentence of its
own.*

Good lines pass. The checks drop an unknown op, an unknown key, a link
and markup:

``` python
test_eq(check_line(dict(op='circle', at=dict(a='m1', tex=r'\bar\alpha_t'), w=0), ctx), '')
test_eq(check_line(dict(op='label', at=dict(a='figure', label='a'), text='side a', side='left'), ctx), '')
test_eq(check_line(dict(op='arrow', **{'from': dict(a='m1', pt=[1100, 500]), 'to': dict(a='m1', tex=r'(1-\bar\alpha_t)')}), ctx), '')
test_eq(check_line(dict(op='clear'), ctx), '')
assert check_line(dict(op='script', code='alert(1)'), ctx)
assert check_line(dict(op='circle', at='passage', html='<b>'), ctx)
assert check_line(dict(op='label', at='passage', text='see https://evil.example'), ctx)
assert check_line(dict(op='label', at='passage', text='<img src=x onerror=alert(1)>'), ctx)
assert check_line(dict(op='step', say='x' * 161), ctx)
```

A sketch marks a part of a figure, with at most 64 points. A sketch on
another anchor drops: a picture of an idea goes in the side panel.

``` python
test_eq(check_line(dict(op='sketch', at=dict(a='figure', pt=[1050, 0]), pts=[[0, 0], [1000, 1000]]), ctx), '')
assert check_line(dict(op='sketch', at='figure', pts=[[i, i] for i in range(65)]), ctx)
test_eq(check_line(dict(op='sketch', at=dict(a='m1', pt=[1050, 0]), pts=[[0, 0], [1000, 1000]]), ctx), 'a sketch that is not on a figure')
```

A circle, a box, an underline or a label marks a thing on the page. On a
bare point it has no thing under it, so it drops. A point and an arrow
can use a point:

``` python
assert 'bare point' in check_line(dict(op='box', at=dict(a='passage', pt=[-200, 900])), ctx)
assert 'bare point' in check_line(dict(op='circle', at=dict(a='passage', pt=[500, 500])), ctx)
test_eq((check_line(dict(op='point', at=dict(a='passage', pt=[-200, 900])), ctx), check_line(dict(op='box', at=dict(a='passage', box=[0, 0, 500, 500])), ctx)), ('', ''))
```

The side panel: a `panel` line with its title, a
[`row`](https://vedicreader.github.io/rupaka/spec.html#row) of short
cells with an optional `to` and anchor, and a `note`. A number in a cell
is good. A long cell, an empty row or an unknown anchor drops:

``` python
test_eq([check_line(o, ctx) for o in (dict(op='panel', title='tensors and targets', head='Make a question and its answer', cols=['Input', 'Target']),
                                       dict(op='row', cells=['[18]'], to=47, at=dict(a='passage', text='closed form')),
                                       dict(op='row', cells=['Guess', 'Compare', 'Adjust'], chain=True), dict(op='note', text='Both can be tensors.'))], [''] * 4)
for bad in (dict(op='row', cells=[]), dict(op='row', cells=['x' * 25]), dict(op='row', cells=['a'], at='e7'), dict(op='panel', cols=['a', 'b', 'c', 'd']),
            dict(op='note', text=''), dict(op='row', cells=['a'], chain='yes')): assert check_line(bad, ctx), bad
```

A flow diagram has boxes with a short id and arrows between them. A bad
id, a long label and a word too long for a cell drop:

``` python
test_eq([check_line(o, ctx) for o in (dict(op='node', id='enc', text='Encoder'), dict(op='node', id='z', text='$\\mathbf{z}$'),
                                       dict(op='edge', **{'from': 'enc', 'to': 'z'}), dict(op='edge', **{'from': 'z', 'to': 'enc', 'text': 'back in'}))], [''] * 4)
test_eq(check_line(dict(op='node', id='1st', text='Encoder'), ctx), 'a bad box id')
test_eq(check_line(dict(op='edge', **{'from': 'a', 'to': 'b', 'text': 'a label that is far too long'}), ctx), 'a bad edge text')
test_eq(check_line(dict(op='row', cells=['Representations!'], to='z'), ctx), '')
test_eq(check_line(dict(op='row', cells=['Autoregressiveness'], to='z'), ctx), 'a word of more than 16 letters in a cell')
test_eq(check_line(dict(op='row', cells=[r'$\\mathrm{autoregressive}$'], to='z'), ctx), '')   # maths does not count
```

A cell can be a blank: `{"blank": value}` hides the value for the
learner to fill. A card of a wrong idea has a short wrong idea, a short
fact and one line of why:

``` python
test_eq(check_line(dict(op='row', cells=['[18, 47]'], to={'blank': 56}), ctx), '')
test_eq(check_line(dict(op='row', cells=[{'blank': {'blank': 1}}], to=2), ctx), 'bad cells')
test_eq(check_line(dict(op='myth', think='Alpha bar grows with t', fact='Alpha bar gets smaller with t', why='Each step keeps less of the picture.'), ctx), '')
test_eq(check_line(dict(op='myth', think='Alpha bar grows with t'), ctx), 'a bad wrong idea')
test_eq(check_line(dict(op='panel', head='From a case to symbols', stages=True), ctx), '')
```

### Repair a panel line

A model often writes a panel line that is a little too long: an arrow
label of a full sentence, a box with a second line, a note with a second
sentence. When such a line drops, the picture breaks: a flow loses an
arrow, and a panel loses its title. So
[`repair_line`](https://vedicreader.github.io/rupaka/board.html#repair_line)
makes the line good when one part of it is bad, and the rest of the line
stays:

- A panel line loses a bad title, head or column names.
- An arrow loses a long label. The arrow is the structure of the flow;
  the label only names it.
- A box keeps the first line of its text, or the name before a colon
  (`Weight: a number to adjust` is `Weight`).
- A long note keeps its first whole sentences that fit. The maths markup
  does not count, only what shows.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L254"
target="_blank" style="float:right; font-size:smaller">source</a>

### repair_line

``` python
def repair_line(
    o, ctx
)->tuple:
```

*`(line, why)`: the side-panel line `o` that
[`check_line`](https://vedicreader.github.io/rupaka/board.html#check_line)
drops, made good (see above), and why; else `(None, '')`.*

An arrow keeps its boxes and loses its long label. A box keeps its name.
A note keeps the sentence that fits:

``` python
test_eq(repair_line(dict(op='edge', **{'from': 'out', 'to': 'dec', 'text': 'Earlier outputs guide the next choice'}), ctx),
        (dict(op='edge', **{'from': 'out', 'to': 'dec'}), 'a long label of an arrow: the label goes'))
test_eq(repair_line(dict(op='node', id='headA', text='Head A\nCompare Q with K, then mix V'), ctx)[0]['text'], 'Head A')
test_eq(repair_line(dict(op='node', id='w', text='Weight: the number that the model adjusts'), ctx)[0]['text'], 'Weight')
note = 'An example with our own numbers. Shops help to explain the roles, but real attention mixes vectors of numbers.'
test_eq(repair_line(dict(op='note', text=note), ctx)[0]['text'], 'An example with our own numbers.')
```

A panel line keeps its title when its head is too long, and a third
column name is good. A good line, and a line with no part to cut, give
None:

``` python
test_eq(repair_line(dict(op='panel', title='Roles', head='Rama serves rice to a child with a spoon from a pot in a garden'), ctx)[0], dict(op='panel', title='Roles'))
test_eq(check_line(dict(op='panel', cols=['Entries', 'Before', 'After']), ctx), '')
test_eq(repair_line(dict(op='note', text='Short.'), ctx), (None, ''))
test_eq(repair_line(dict(op='note', text='x' * 120), ctx), (None, ''))
test_eq(check_line(dict(op='note', text='$e^{1.414}\\approx4.113$ and $e^0=1$. Divide each by $5.113$ to get the shares of the two.'), ctx), '')   # 88 characters show
```

The demo gate: `demo_set` drops while the learner has not locked in a
prediction, so the board cannot give the result first. After the lock-in
it changes only inputs that the demo has, in their range.

``` python
assert 'gate' in check_line(dict(op='demo_set', set=dict(t=500)), ctx)
open_ctx = by_id({**ctx, 'demo': {**ctx.demo, 'open': True}})
test_eq(check_line(dict(op='demo_set', set=dict(t=500)), open_ctx), '')
assert check_line(dict(op='demo_set', set=dict(beta=1)), open_ctx)
assert check_line(dict(op='demo_set', set=dict(t=5000)), open_ctx)
```

`video_at` quotes the transcript at its time, word for word:

``` python
vctx = by_id({**ctx, 'video': dict(id='abc123def45', cues=[[0, 'Today we add noise.'], [61.5, 'Alpha bar is the share of the signal that is left.'], [70, 'It falls as t grows.']])})
test_eq(check_line(dict(op='video_at', t=63, quote='the share of the signal'), vctx), '')
assert check_line(dict(op='video_at', t=63, quote='the share of the noise'), vctx)
assert check_line(dict(op='video_at', t=63, quote='Alpha bar'), vctx)      # too short to be a quote
assert check_line(dict(op='video_at', t=63, quote='the share of the signal'), ctx)   # no video on the page
```

[`cue_quote`](https://vedicreader.github.io/rupaka/board.html#cue_quote)
gives the start of the cue that holds the quote, and the words as the
transcript has them:

``` python
cs = vctx.video['cues']
test_eq(cue_quote(cs, 63, 'THE SHARE of the signal'), (61.5, 'the share of the signal'))
test_eq(cue_quote(cs, 71, 'share of the signal'), (61.5, 'share of the signal'))   # the time is late: the cue before holds it
test_eq(cue_quote(cs, 63, 'left. It falls'), (61.5, 'left. It falls'))             # a quote across two cues starts in the first
test_eq(cue_quote(cs, 5, 'the share of the signal'), (61.5, 'the share of the signal'))
assert cue_quote(cs, 200, 'Today we add noise.') is None                           # too far from its time
```

### The whole reply

[`parse_board`](https://vedicreader.github.io/rupaka/board.html#parse_board)
reads a reply line by line. Each step keeps at most 6 strokes, and a
board has at most 6 steps. A stroke before the first step drops. Two
dropped lines in a row end the board (`failed`): the steps before them
stay. A dropped line of the side panel does not count for this: it is a
long label or a long text, not a reply that went wrong.

A step can quote a source: `src` is the number of the source in
`ctx.sources`, and `quote` must be word for word in its text, with 40 to
600 characters.
[`verbatim`](https://vedicreader.github.io/rupaka/board.html#verbatim)
checks it: white space is made single, and the quote goes from a word
boundary to a word boundary. A host can give its own check as
`verbatim(quote, text) -> str` (abhyasa gives the rules of its sources).
A quote that fails drops, and the step stays, in the tutor’s words.

### Each row with its step

A step says one thing, and its row and its marks show that same thing.
So the spoken line of a step must name what its row shows.
[`key_terms`](https://vedicreader.github.io/rupaka/board.html#key_terms)
gives the key words and symbols of a text: each word of 3 letters or
more that is not a small word, each number, and each maths token such as
`x_i`. [`named`](https://vedicreader.github.io/rupaka/board.html#named)
is true when a row (or a note) and a spoken line share one of them. The
check is cheap and knows no subject.
[`page_terms`](https://vedicreader.github.io/rupaka/board.html#page_terms)
gives the key terms of what the page shows.

The panel is an example. It can come from the page, or it can be the
tutor’s own case (“deva + ālaya → devālaya” on a page that names no
word). So
[`parse_board`](https://vedicreader.github.io/rupaka/board.html#parse_board)
drops a row that names no word of the page only when another row of its
panel does, and when it shares no word with the other rows or the title
and head of its panel: then it is the odd one out, such as a later idea
at the end of a worked example. A row with only numbers or symbols, and
a box of a flow diagram, never drop for this: a diagram with one box
less is broken. Then it pairs each row, note and box with its step
([`_pair`](https://vedicreader.github.io/rupaka/board.html#_pair)). A
row that its own step does not name moves to the next step of its panel
that names it. A row that no step names drops. A row never moves back to
an earlier step, so a stream sends each row once, with the step that
explains it.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L303"
target="_blank" style="float:right; font-size:smaller">source</a>

### page_terms

``` python
def page_terms(
    ctx
)->set:
```

*The key terms of what the page shows: the text, tokens and labels of
each anchor, the sources, and the demo.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L299"
target="_blank" style="float:right; font-size:smaller">source</a>

### named

``` python
def named(
    o, say:str
)->bool:
```

*Whether the spoken line `say` names the panel line `o` (a row or a
note): they share a key word or a symbol.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L278"
target="_blank" style="float:right; font-size:smaller">source</a>

### key_terms

``` python
def key_terms(
    s
)->set:
```

*The key words and symbols of `s`: each word of 3 letters or more that
is not a small word, cut to its first 5 letters; each* number; each
maths token such as `x_i` or `w_1x_1`; each single letter other than “a”
and “I”. `\sum` counts as the word sum.

### One shape for each panel

A panel is a table, a list or a flow diagram. Its lines line up only
when they have one shape. A row is a *pair* (one or more cells, then an
arrow and the answer `to`) or a *chain* (words with an arrow between
each two).
[`_shape`](https://vedicreader.github.io/rupaka/board.html#_shape) sets
the shape:

- **A flow diagram**: the first line of the panel is a box (`node`). Its
  boxes and arrows (`edge`) stay, and rows drop. An arrow goes right
  after the later of its two boxes
  ([`_edges`](https://vedicreader.github.io/rupaka/board.html#_edges)),
  and an arrow to a box that is not there drops.
- **A table**: the panel has column names `cols`, and its first two rows
  are pairs with one cell less than the names and an answer. The names
  stand over the cells and the answers, so a later row that does not fit
  them drops. A chain of two words fits as one cell and its answer.
- **A list**: else the names go and every row stays. A list lines up its
  chains on one grid, and the answer of a pair goes in the last column.
  Two short names say less than the rows, so the rows win.
- **A card of wrong ideas**: the first line is a `myth`. It takes at
  most 2 of them, and no rows or boxes.
- **Three stages**: a panel with `stages` is a list of at most 3 rows: a
  case, a picture, then the symbols. It has no column names.
- **A faded worked example**: a row can hide parts as blanks. The first
  row of a panel shows its blanks, because it is the worked one.
- At most 6 rows, 6 boxes and 8 arrows. More do not fit next to the
  page.

The first two rows decide, and not all of them, because a stream sends
each row with its step: a later row cannot change the rows that show. So
a stream holds a panel with column names until its second row is in
([`_held`](https://vedicreader.github.io/rupaka/board.html#_held)).

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L482"
target="_blank" style="float:right; font-size:smaller">source</a>

### parse_board

``` python
def parse_board(
    text:str, ctx, verbatim:function=verbatim
)->fastcore.basics.AttrDict:
```

*The checked board in the model’s reply `text`: `steps`, `dropped` lines
with why, `fixed` lines with why
([`repair_line`](https://vedicreader.github.io/rupaka/board.html#repair_line)),
`hard` spoken lines with their
[`plain_why`](https://vedicreader.github.io/rupaka/spec.html#plain_why)
problems, `sure`, `need` and `failed`.
[`verbatim`](https://vedicreader.github.io/rupaka/board.html#verbatim)
checks a quote.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L426"
target="_blank" style="float:right; font-size:smaller">source</a>

### verbatim

``` python
def verbatim(
    quote:str, text:str
)->str:
```

*`quote` with its white space made single, when it is in `text` word for
word, from a word boundary to a word boundary, and has 40 to 600
characters; else ’’.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L422"
target="_blank" style="float:right; font-size:smaller">source</a>

### yt_link

``` python
def yt_link(
    vid:str, t:float
)->str:
```

*The watch link that opens the video `vid` at `t` seconds, as fossick’s
[`yt_link`](https://vedicreader.github.io/rupaka/board.html#yt_link)
writes it.*

The default check of a quote: white space does not count, a part of a
word does not count, and a short quote does not count:

``` python
text = 'A notable property of the forward process is that it admits sampling x_t at an arbitrary timestep t in closed form.'
test_eq(verbatim('it admits  sampling x_t at an arbitrary timestep t in closed form', text), 'it admits sampling x_t at an arbitrary timestep t in closed form')
test_eq(verbatim('t admits sampling x_t at an arbitrary timestep t in closed form', text), '')
test_eq(verbatim('closed form', text), '')
```

A board for “I don’t get alpha bar” on the DDPM formula: three steps,
each with its strokes. The header says that the tutor is sure.

``` python
reply = '\n'.join(json.dumps(o) for o in [
    dict(op='v', n=1, sure=0.9, need='none'),
    dict(op='step', say='This formula adds noise in one jump.'), dict(op='box', at='m1', w=2),
    dict(op='step', say='Alpha bar t is how much of the clean picture is left at step t.'),
    dict(op='circle', at=dict(a='m1', tex=r'\bar\alpha_t'), w=0), dict(op='label', at=dict(a='m1', tex=r'\bar\alpha_t'), text='how much is left', side='above', w=5),
    dict(op='step', say='It starts near one and falls to near zero.'), dict(op='sketch', at=dict(a='figure', pt=[1050, 0]), pts=[[0, 100], [500, 500], [1000, 950]])])
b = parse_board(reply, ctx)
test_eq(len(b.steps), 3)
test_eq([len(s['strokes']) for s in b.steps], [1, 2, 1])
test_eq((b.sure, b.need, b.failed, b.dropped), (0.9, 'none', False, []))
```

A bad line drops alone, and the next line still draws. Two bad lines in
a row end the board:

``` python
b = parse_board('{"op":"step","say":"One."}\n{"op":"circle","at":"e7"}\n{"op":"circle","at":"m1"}', ctx)
test_eq((len(b.steps[0]['strokes']), len(b.dropped), b.failed), (1, 1, False))
b = parse_board('{"op":"step","say":"One."}\nnot json\n{"op":"circle","at":"e7"}\n{"op":"step","say":"Two."}', ctx)
test_eq((len(b.steps), b.failed), (1, True))
```

A spoken line that is hard to read stays, and `hard` names its problems
([`plain_why`](https://vedicreader.github.io/rupaka/spec.html#plain_why)),
with no model call: the host can ask the next board for plainer lines. A
plain line gives none:

``` python
b = parse_board('{"op":"step","say":"This departs from the rule."}\n{"op":"step","say":"Alpha bar is the part of the picture that is left."}', ctx)
test_eq((len(b.steps), [h['n'] for h in b.hard]), (2, [1]))
assert any('"departs from" is vague' in w for w in b.hard[0]['why'])
```

Two bad panel lines in a row do not end the board. They are a long box
and a box with a bad id, not a reply that went wrong:

``` python
two = '\n'.join(json.dumps(o) for o in [dict(op='step', say='One.'), dict(op='panel', head='Parts'), dict(op='node', id='a', text='x' * 40),
                                        dict(op='node', id='1b', text='B'), dict(op='step', say='Two.'), dict(op='circle', at='m1')])
b = parse_board(two, ctx)
test_eq((len(b.steps), b.failed), (2, False))
```

A step that talks about the board, not the idea, drops with its strokes,
such as an excuse for an earlier offer. It does not end the board:

``` python
meta = '\n'.join(json.dumps(o) for o in [dict(op='step', say='You asked to see it move, but this page has no live demo I can start; my earlier offer was misleading.'),
                                         dict(op='box', at='passage'), dict(op='step', say='Alpha bar t is how much is left.'), dict(op='circle', at='m1')])
b = parse_board(meta, ctx)
test_eq(([s['say'] for s in b.steps], b.steps[0]['strokes'], b.failed), (['Alpha bar t is how much is left.'], [dict(op='circle', at='m1')], False))
test_eq([w for _, w in b.dropped], [META_WHY, 'a stroke of a dropped step'])
test_eq([bool(META.search(s)) for s in ('Sorry, I cannot show that.', 'I apologise for the mix-up.', 'The noise is not misleading here.', 'The last step adds noise.')], [True, True, False, False])
```

A row with `at` draws its own circle, so a circle line on the same thing
drops. A row comes to the page with its cells as text. A semicolon in a
spoken line becomes the end of a sentence:

``` python
both = '\n'.join(json.dumps(o) for o in [dict(op='step', say='Alpha bar sets the mix; 18 goes in and 47 comes out.'), dict(op='circle', at=dict(a='passage', text='closed form')),
                                         dict(op='row', cells=[18], to=47, at=dict(a='passage', text='closed form'))])
b = parse_board(both, ctx)
test_eq((b.steps[0]['say'], b.steps[0]['strokes'], b.dropped), ('Alpha bar sets the mix. 18 goes in and 47 comes out.', [dict(op='row', cells=['18'], to='47', at=dict(a='passage', text='closed form'))],
                                                               [(-1, 'the row draws its own circle')]))
```

A row of a worked example with the answer in a last cell, and no `to`,
gets its answer after the arrow. A chain stays a chain, and a pair keeps
its answer next to it (see “One shape for each panel” below). A head
that names “four stages” over fewer rows loses the count:

``` python
b2 = parse_board('\n'.join(json.dumps(o) for o in [dict(op='step', say='Work it out from 1.0: guess, compare and update the weights.'), dict(op='panel', head='Repeat these four stages', numbered=True),
    dict(op='row', cells=['1.0 and 1.0', '√0.5 × 1 + √0.5 × 1', '1.414']), dict(op='panel', head='Then learn'), dict(op='row', cells=['Guess', 'Compare'], chain=True), dict(op='row', cells=['Update'], to='weights')]), ctx)
rs = [s for s in b2.steps[0]['strokes'] if s['op'] == 'row']
test_eq([(r['cells'], r.get('to'), r.get('chain', False)) for r in rs], [(['1.0 and 1.0', '√0.5 × 1 + √0.5 × 1'], '1.414', False), (['Guess', 'Compare'], None, True), (['Update'], 'weights', False)])
test_eq(b2.steps[0]['strokes'][0]['head'], 'Repeat these stages')
```

The key terms of a line: words cut to 5 letters, numbers and maths
tokens. A small word and the article “a” do not count:

``` python
test_eq(key_terms('Multiply each $x_i$ by $w_i$ to get w_1x_1 and 47'), {'multi', 'x_i', 'w_i', 'w_1x_1', '47'})
test_eq(key_terms(r'Add the products: \\sum of a list'), {'add', 'produ', 'sum', 'list'})
```

A row is named when its step shares a word (in any form) or a symbol
with it. The single letter x does not name `x_i`:

``` python
test_eq(named(dict(op='row', cells=['Add products, then b'], to='z'), 'Then add the products and the bias b.'), True)
test_eq(named(dict(op='row', cells=['Multiply each x_i by w_i'], to='w_1x_1, …, w_nx_n'), 'Numbers x go in, one number z comes out.'), False)
```

The reply of the screenshot that a tester sent, on the idea “Functions
and the neuron”: step 1 says that numbers go in and one number comes
out, but its row multiplies each input by its weight. After the check,
that row moves to step 2, whose line names it. Step 1 keeps its circle,
because a stream sends that circle before the row comes:

``` python
nctx = by_id(dict(anchors=[dict(id='passage', kind='passage', text='A neuron takes numbers x_1, ..., x_n, multiplies each by a weight w_i, adds a bias b, and outputs z = w_1x_1 + ... + w_nx_n + b.',
                                tokens=['x_1', 'w_i', 'b', 'z', 'w_1x_1'])]))
w1 = dict(a='passage', tex='w_1x_1')
neuron = [dict(op='v', n=1, sure=0.9, need='none'),
    dict(op='step', say='This page asks you to see a neuron as a function: numbers x go in, one number z comes out.'),
    dict(op='panel', title='Inside one neuron', head='Turn many inputs into one output', cols=['Step', 'Result'], numbered=True),
    dict(op='circle', at=w1), dict(op='row', cells=['Multiply each x_i by w_i'], to='w_1x_1, …, w_nx_n', at=w1, w=4),
    dict(op='step', say='First it multiplies each input by its weight, then adds the products and the bias b.'),
    dict(op='row', cells=['Add products, then b'], to='z'),
    dict(op='step', say='The output z then goes on to the next layer.'),
    dict(op='row', cells=['Layer 1 outputs z'], to='Layer 2 inputs x')]
b = parse_board('\n'.join(json.dumps(o) for o in neuron), nctx)
test_eq([[s['op'] for s in st['strokes']] for st in b.steps], [['panel', 'circle'], ['row', 'row'], ['row']])
test_eq(b.steps[1]['strokes'][0], dict(op='row', cells=['Multiply each x_i by w_i'], to='w_1x_1, …, w_nx_n', at=w1))
```

Here “z” names the row of step 3, so it stays. With a line that names
nothing of it, the row drops:

``` python
neuron[-2] = dict(op='step', say='That is the whole neuron.')
b = parse_board('\n'.join(json.dumps(o) for o in neuron), nctx)
test_eq(([[s['op'] for s in st['strokes']] for st in b.steps], (-1, 'a row that its step does not name') in b.dropped), ([['panel', 'circle'], ['row', 'row'], []], True))
```

A well-formed reply, where each line names its row, stays as it is:

``` python
good_n = [neuron[0], dict(op='step', say='A neuron is a function: numbers x go in, and one number z comes out.'),
    dict(op='panel', title='Inside one neuron', head='Turn many inputs into one output', numbered=True),
    dict(op='step', say='First it multiplies each input x i by its weight w i.'), dict(op='row', cells=['Multiply each $x_i$ by $w_i$'], to='$w_1x_1, …, w_nx_n$', at=w1),
    dict(op='step', say='Then it adds the products and the bias b, and that sum is z.'), dict(op='row', cells=['Add the products and $b$'], to='$z$')]
b = parse_board('\n'.join(json.dumps(o) for o in good_n), nctx)
test_eq((b.dropped, [[s['op'] for s in st['strokes']] for st in b.steps]), ([], [['panel'], ['row'], ['row']]))
```

A row that names nothing on the page drops when another row of its panel
is on the page, also when its line names it: a second layer and its
softmax are later ideas on a page about one neuron.

``` python
later = [dict(op='step', say='It multiplies each input x i by its weight w i.'), dict(op='panel', head='Inside'), dict(op='row', cells=['Multiply each x_i by w_i'], to='w_1x_1'),
         dict(op='step', say='Layer 2 then runs a softmax.'), dict(op='row', cells=['Layer 2'], to='softmax')]
b = parse_board('\n'.join(json.dumps(o) for o in later), nctx)
test_eq(([[s['op'] for s in st['strokes']] for st in b.steps], b.dropped), ([['panel', 'row'], []], [(-1, 'a row that names nothing on the page')]))
```

The tutor’s own case stays when no row of its panel is on the page, and
so does a row of only numbers:

``` python
own = [dict(op='step', say='Deva and alaya join to make devalaya: a and a become one long a.'), dict(op='panel', head='Join two words'),
       dict(op='row', cells=['deva + ālaya'], to='devālaya'), dict(op='row', cells=['deva + iti'], to='deveti')]
b = parse_board('\n'.join(json.dumps(o) for o in own), nctx)
test_eq(([s['op'] for s in b.steps[0]['strokes']], b.dropped), (['panel', 'row', 'row'], []))
mixed = own[:2] + [dict(op='row', cells=['Join two vowels'], to='one long vowel'), *own[2:]]
mixed[0] = {**mixed[0], 'say': 'Deva and alaya join to make devalaya: two vowels join into one long vowel.'}
nctx2 = by_id({**nctx, 'anchors': [*nctx.anchors, dict(id='rule', kind='text', text='Like vowels join into one long vowel.')]})
test_eq([s['op'] for s in parse_board('\n'.join(json.dumps(o) for o in mixed), nctx2).steps[0]['strokes']], ['panel', 'row', 'row', 'row'])   # own rows that belong together stay
```

A row does not move past the next panel line, because that line starts a
new example. Here the row drops, and the two panels have nothing in
them, so they go too:

``` python
two = [dict(op='step', say='Numbers go in.'), dict(op='panel', head='One'), dict(op='row', cells=['Multiply'], to='w_1x_1'),
       dict(op='step', say='A new example.'), dict(op='panel', head='Two'), dict(op='step', say='Multiply them.')]
test_eq([[s['op'] for s in st['strokes']] for st in parse_board('\n'.join(json.dumps(o) for o in two), ctx).steps], [[], [], []])
```

The board of a screenshot from the owner, on the step “Auto-regressive
encoder-decoder transduction” of “Attention Is All You Need”. Its panel
has the column names “From” and “To”, and its rows have three shapes:
pairs, chains of three words, and a note. The names stood over no cells,
and nothing lined up. A small context with the passage and its formulas:

``` python
actx = by_id(dict(anchors=[dict(id='passage', kind='passage', text='Here, the encoder maps an input sequence of symbol representations $(x_1, \dots, x_n)$ to a sequence of continuous representations $\mathbf{z} = (z_1, \dots, z_n)$. Given $\mathbf{z}$, the decoder then generates an output sequence $(y_1, \dots, y_m)$ of symbols one element at a time.'),
                           dict(id='m1', kind='formula', text=r'(x_1, \dots, x_n)', tokens=['x_1', 'x_n'])]))
att = [dict(op='step', say='The encoder turns the input symbols x_1 to x_n into a list z.'),
       dict(op='panel', title='Turn input into output', head='Symbols to symbols', cols=['From', 'To']),
       dict(op='row', cells=['Input symbols'], to='Output symbols'), dict(op='row', cells=[r'$x_1,\dots,x_n$', 'Encoder', r'$z=(z_1,\dots,z_n)$'], chain=True),
       dict(op='step', say='The decoder reads z and gives the output symbols y_1 to y_m.'), dict(op='row', cells=['$z$', 'Decoder', r'$y_1,\dots,y_m$'], chain=True),
       dict(op='step', say='Earlier outputs go back into the decoder, so they help shape the next output.'),
       dict(op='row', cells=['Earlier outputs'], to='Decoder'), dict(op='note', text='Earlier outputs help shape the next output.')]
rows = lambda b: [(r['cells'], r.get('to'), r.get('chain', False)) for st in b.steps for r in st['strokes'] if r['op'] == 'row']
```

The first row is a pair that fits the two names, but the second is a
chain. So the names go, and the panel is a list: every row stays, and
the pairs keep their answers:

``` python
b = parse_board('\n'.join(json.dumps(o) for o in att), actx)
test_eq([(len(c), to) for c, to, chain in rows(b)], [(1, 'Output symbols'), (3, None), (3, None), (1, 'Decoder')])
test_eq(('cols' in b.steps[0]['strokes'][0], b.dropped), (False, [(-1, 'column names that do not fit the rows')]))
```

When the first two rows fit the names, the panel is a table. A later
chain of three words does not fit the names on the page, so it drops:

``` python
table = [*att[:3], att[6], att[7], att[4], att[5]]
b = parse_board('\n'.join(json.dumps(o) for o in table), actx)
test_eq(rows(b), [(['Input symbols'], 'Output symbols', False), (['Earlier outputs'], 'Decoder', False)])
test_eq((b.steps[0]['strokes'][0]['cols'], b.dropped), (['From', 'To'], [(-1, 'a row that does not fit the column names')]))
```

A chain of two words fits a table as one cell and its answer. A pair
with two cells does not fit a table of pairs with one cell:

``` python
pp = [dict(op='step', say='Guess, then compare the guess.'), dict(op='panel', head='Guess and compare', cols=['Guess', 'Compare']),
      dict(op='row', cells=['Guess'], to='compare'), dict(op='row', cells=['Guess', 'Compare'], chain=True), dict(op='row', cells=['Guess', 'Compare'], to='update')]
b = parse_board('\n'.join(json.dumps(o) for o in pp), ctx)
test_eq(rows(b), [(['Guess'], 'compare', False), (['Guess'], 'Compare', False)])
test_eq(b.dropped, [(-1, 'a row that does not fit the column names')])
```

A panel shows at most 6 rows. The seventh drops:

``` python
seven = [dict(op='step', say='Guess and compare.'), dict(op='panel', head='Guess'), *[dict(op='row', cells=['Guess'], to=f'compare {k}') for k in range(4)],
         dict(op='step', say='Guess again and compare.'), *[dict(op='row', cells=['Guess'], to=f'compare {k}') for k in range(4, 7)]]
b = parse_board('\n'.join(json.dumps(o) for o in seven), ctx)
test_eq((len(rows(b)), b.dropped), (6, [(-1, 'more than 6 rows in a panel')]))
```

The same idea as a flow diagram: the input goes to the encoder, the
encoder gives **z**, the decoder reads it and gives the output, and the
output goes back into the decoder. One or two boxes come in each step,
and an arrow can come before its box: it moves to the line after that
box.

``` python
fl = [dict(op='step', say='The encoder turns the input symbols into a list z.'), dict(op='panel', title='Encoder and decoder', head='Symbols to symbols'),
      dict(op='node', id='x', text='Input symbols'), dict(op='node', id='enc', text='Encoder'), dict(op='edge', **{'from': 'x', 'to': 'enc'}),
      dict(op='edge', **{'from': 'enc', 'to': 'z'}), dict(op='node', id='z', text='$\\mathbf{z}$'),
      dict(op='step', say='The decoder reads z and gives the output symbols one at a time.'), dict(op='node', id='dec', text='Decoder'),
      dict(op='edge', **{'from': 'z', 'to': 'dec'}), dict(op='node', id='y', text='Output symbols'), dict(op='edge', **{'from': 'dec', 'to': 'y'}),
      dict(op='step', say='Each output goes back into the decoder for the next one.'), dict(op='edge', **{'from': 'y', 'to': 'dec', 'text': 'earlier outputs'})]
b = parse_board('\n'.join(json.dumps(o) for o in fl), actx)
ops = lambda st: [s['op'] + ':' + (s.get('id') or f"{s.get('from')}>{s.get('to')}") for s in st['strokes'] if s['op'] in FLOW]
test_eq([ops(st) for st in b.steps], [['node:x', 'node:enc', 'edge:x>enc', 'node:z', 'edge:enc>z'], ['node:dec', 'edge:z>dec', 'node:y', 'edge:dec>y'], ['edge:y>dec']])
test_eq(b.dropped, [])
```

A flow diagram takes no rows and no column names. An arrow to a box that
never comes, and a second box with the same id, drop:

``` python
bad = [*fl[:3], dict(op='row', cells=['Input'], to='Encoder'), dict(op='edge', **{'from': 'x', 'to': 'q'}), dict(op='node', id='x', text='Input again')]
bad[1] = {**bad[1], 'cols': ['From', 'To']}
b = parse_board('\n'.join(json.dumps(o) for o in bad), actx)
test_eq([s['op'] for s in b.steps[0]['strokes']], ['panel', 'node'])
test_eq(sorted(w for _, w in b.dropped), ['a box with the id of another box', 'a row in a panel of a flow diagram', 'an arrow to a box that is not there', 'column names that do not fit the rows'])
```

A faded worked example: the first row is solved, so its blank shows.
Each later row hides one more part, and the blanks stay:

``` python
fade = [dict(op='step', say='Guess, compare and update: 1.0 gives 1.414.'), dict(op='panel', head='Guess, compare, update', cols=['Guess', 'Compare']),
        dict(op='row', cells=['1.0'], to={'blank': '1.414'}), dict(op='step', say='Now you guess for 2.0: compare and update.'), dict(op='row', cells=['2.0'], to={'blank': '2.0'}),
        dict(op='step', say='Then you guess, compare and update the weights.'), dict(op='row', cells=[{'blank': 'Guess'}], to={'blank': 'update the weights'})]
b = parse_board('\n'.join(json.dumps(o) for o in fade), ctx)
test_eq([(r['cells'], r['to']) for st in b.steps for r in st['strokes'] if r['op'] == 'row'],
        [(['1.0'], '1.414'), (['2.0'], {'blank': '2.0'}), ([{'blank': 'Guess'}], {'blank': 'update the weights'})])
test_eq(b.dropped, [])
```

A card of wrong ideas takes at most two, and no row:

``` python
my = [dict(op='step', say='Alpha bar does not grow with t: it gets smaller.'), dict(op='panel', head='A common wrong idea'),
      dict(op='myth', think='Alpha bar grows with t', fact='Alpha bar gets smaller with t', why='Each step keeps less of the picture.'),
      dict(op='row', cells=['alpha bar'], to='smaller'), dict(op='myth', think='alpha bar is noise', fact='alpha bar is the share of signal'),
      dict(op='myth', think='alpha bar is t', fact='alpha bar depends on t')]
b = parse_board('\n'.join(json.dumps(o) for o in my), ctx)
test_eq([s['op'] for s in b.steps[0]['strokes']], ['panel', 'myth', 'myth'])
test_eq([w for _, w in b.dropped], ['a row in a panel of wrong ideas', 'more than 2 wrong ideas in a panel'])
```

A wrong idea is often not on the page, and the step may not name its
words. It stays with its step:

``` python
off = [dict(op='step', say='A common mix-up is how it learns.'), dict(op='panel', head='A common mix-up'),
       dict(op='myth', think='It reads the answer while it learns', fact='It sees only its own past steps', why='No answer is there yet.')]
b = parse_board('\n'.join(json.dumps(o) for o in off), ctx)
test_eq([s['op'] for s in b.steps[0]['strokes']], ['panel', 'myth'])
```

When each line in a panel drops, the panel line drops too, so the page
shows no empty card:

``` python
bad = [dict(op='step', say='A common mix-up.'), dict(op='panel', head='A common mix-up'), dict(op='myth', think='x' * 70, fact='y')]
b = parse_board('\n'.join(json.dumps(o) for o in bad), ctx)
test_eq((b.steps[0]['strokes'], [w for _, w in b.dropped]), ([], ['a bad wrong idea', 'a panel with nothing in it']))
```

Three stages: at most three rows, and no column names:

``` python
sg = [dict(op='step', say='From 1.0 to 1.414: guess, compare and update, as alpha bar does in the formula.'), dict(op='panel', head='From a case to the symbols', stages=True, cols=['a', 'b']),
      dict(op='row', cells=['1.0'], to='1.414'), dict(op='row', cells=['Guess', 'Compare', 'Update'], chain=True), dict(op='row', cells=['$\\bar\\alpha_t$'], to='$\\sqrt{\\bar\\alpha_t}$'),
      dict(op='row', cells=['update'], to='weights')]
b = parse_board('\n'.join(json.dumps(o) for o in sg), ctx)
test_eq((len(rows(b)), 'cols' in b.steps[0]['strokes'][0]), (3, False))
test_eq([w for _, w in b.dropped], ['column names that do not fit the rows', 'more than 3 rows in a panel'])
```

Some models put the strokes inside the step. They count as the lines
after it:

``` python
b = parse_board('{"op":"step","say":"One.","strokes":[{"op":"circle","at":"m1"},{"op":"box","at":"m1"}]}', ctx)
test_eq(([s['op'] for s in b.steps[0]['strokes']], b.dropped), (['circle', 'box'], []))
```

At most 6 strokes in a step and 6 steps in a board; a stroke before the
first step drops. An arrow of a flow diagram does not count, because it
only joins two boxes:

``` python
many = '\n'.join(['{"op":"step","say":"One."}'] + ['{"op":"circle","at":"m1"}'] * 6 + ['{"op":"box","at":"m1"}', '{"op":"step","say":"Two."}'])
b = parse_board(many, ctx)
test_eq(([len(s['strokes']) for s in b.steps], len(b.dropped)), ([6, 0], 1))
test_eq(len(parse_board('\n'.join(['{"op":"step","say":"A step."}'] * 7), ctx).steps), 6)
test_eq(parse_board('{"op":"circle","at":"m1"}\n{"op":"step","say":"One."}', ctx).dropped[0][1], 'a stroke before the first step')
```

The count of steps follows the reply: a board of 1, 2 or 5 steps keeps
that count. Only a seventh step drops:

``` python
def steps(n): return '\n'.join(json.dumps(dict(op='step', say=f'Part {k} of the idea.')) for k in range(1, n + 1))
test_eq([len(parse_board(steps(n), ctx).steps) for n in (1, 2, 5, 7)], [1, 2, 5, 6])
```

A quote stays only when it is word for word in its source:

``` python
qctx = by_id({**ctx, 'sources': [dict(title='DDPM', text='A notable property of the forward process is that it admits sampling x_t at an arbitrary timestep t in closed form.')]})
good = dict(op='step', say='The paper says it.', src=1, quote='it admits sampling x_t at an arbitrary timestep t in closed form')
test_eq(parse_board(json.dumps(good), qctx).steps[0]['quote']['src'], 1)
b = parse_board(json.dumps({**good, 'quote': 'it allows sampling x_t at any timestep t in closed form, fast'}), qctx)
assert 'quote' not in b.steps[0] and b.dropped
```

A host can give its own check of a quote. This one takes no quote:

``` python
test_eq('quote' in parse_board(json.dumps(good), qctx, verbatim=lambda q, t: '').steps[0], False)
```

A `video_at` stroke moves to the start of its cue and gets the link to
that time:

``` python
b = parse_board('\n'.join([json.dumps(dict(op='step', say='The video says it.')), json.dumps(dict(op='video_at', t=66, quote='share of the SIGNAL', w=2))]), vctx)
test_eq(b.steps[0]['strokes'], [dict(op='video_at', t=61.5, quote='share of the signal', link='https://www.youtube.com/watch?v=abc123def45&t=61s', w=2)])
```

A line of more than 600 bytes drops. There is no diagram: a picture of
an idea is a side panel, so a diagram line is an unknown op:

``` python
test_eq(parse_board(json.dumps(dict(op='step', say='Long.')) + '\n' + json.dumps(dict(op='label', at='passage', text='x' * 40, side='left' + ' ' * 700)), ctx).dropped[0][1],
        'not one JSON object of 600 bytes or less')
test_eq([w for _, w in parse_board(json.dumps(dict(op='step', say='A tree.')) + '\n' + json.dumps(dict(op='diagram', nodes=[], edges=[])), ctx).dropped], ['an unknown op'])
```

### The context

The page sends what it shows: its address, the step, the concept, the
anchors, the demo, the video, and what the learner showed with the
pointer (the focus), and the thread: the last questions on this page.
[`clean_context`](https://vedicreader.github.io/rupaka/board.html#clean_context)
keeps only the keys that it knows and cuts the text: at most 40 anchors,
300 characters an anchor and 6000 in all. An anchor id has only letters,
digits and `:_-`. An unknown kind becomes `text`. A `try` anchor is the
last graded try of the open check (see *The learner’s answer*): it keeps
its own fields, each with its own bound (`TRY`). The result is a
[`PageContext`](https://vedicreader.github.io/rupaka/board.html#pagecontext).
A host cleans each context that is not one (abhyasa’s `explain` does),
so a raw dict is cleaned too, even when it has a `by` key.

#### What the learner shows

While the learner talks or types the question, they can circle a part
with the mouse, or hold the pointer still on it. The host finds the
gestures and sends them as `focus`: a list of items, the latest first.
Each item has a `kind` (`circle` or
[`point`](https://vedicreader.github.io/rupaka/spec.html#point)), the
`anchors` under it (the best one first), a `box` from 0 to 1000 of its
first anchor, and `t`, the milliseconds from the start of the question.
An item can have `words`: what the learner said or typed near that time.

[`_focus`](https://vedicreader.github.io/rupaka/board.html#_focus) keeps
at most 5 good items. It keeps only the anchor ids that the page sent,
and drops an item with none of them, with a bad kind or with a bad box.

#### The thread of a page

A learner who asks “say that more simply” or “why?” after a board means
that board. So the page keeps a short thread in memory, and sends it
with the next question: the last 3 turns on this page, each with the
question (`said`), the lines that its board said (`says`) and its tier
(`t`). The page forgets the thread when the page changes, after 10
minutes with no question, or when the learner pushes Start fresh. The
server keeps no thread: each question brings its own.

[`_thread`](https://vedicreader.github.io/rupaka/board.html#_thread)
keeps the good turns, the oldest first: a question of at most 300
characters, at most 6 lines of at most 160 characters, and a known tier.
A turn with no question or no line drops, and so does a turn with a tag
or a link in a line, which a board never says. At most 1500 characters
stay in all: the oldest turn goes first.

The demo part keeps the inputs (with their value, range and label), the
readouts, and whether the gate is open.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L615"
target="_blank" style="float:right; font-size:smaller">source</a>

### clean_context

``` python
def clean_context(
    raw
)->__main__.PageContext:
```

*The page context `raw`, with only known keys, cut to size, and with
`by` (its anchors by id).*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L612"
target="_blank" style="float:right; font-size:smaller">source</a>

### PageContext

``` python
def PageContext(
    *args, **kwargs
):
```

*A page context that
[`clean_context`](https://vedicreader.github.io/rupaka/board.html#clean_context)
made. A host cleans every other context, also one with a `by` key.*

Unknown keys go, and a bad anchor id goes:

``` python
c = clean_context(dict(page='/walk/x/step/1', onload='x', anchors=[dict(id='passage', kind='passage', text='Some  text.'), dict(id='<b>', text='x')]))
test_eq(([a['id'] for a in c.anchors], c.anchors[0]['text'], 'onload' in c), (['passage'], 'Some text.', False))
```

A raw dict with a `by` key is not a
[`PageContext`](https://vedicreader.github.io/rupaka/board.html#pagecontext),
so a host cleans it. A cleaned context stays the same type:

``` python
test_eq((isinstance(c, PageContext), isinstance(dict(c), PageContext)), (True, False))
```

A circle on the formula and a point on the passage stay, the latest
first. An unknown anchor id goes from its item:

``` python
shown = [dict(kind='circle', box=[100, 50, 400, 900], anchors=['m1', 'e7', 'passage'], t=2100.4, words='why is this  here'),
         dict(kind='point', box=[480, 480, 520, 520], anchors=['passage'], t=800)]
c = clean_context(dict(anchors=ctx.anchors, focus=shown))
test_eq(c.focus, [dict(kind='circle', box=[100, 50, 400, 900], anchors=['m1', 'passage'], t=2100, words='why is this here'),
                  dict(kind='point', box=[480, 480, 520, 520], anchors=['passage'], t=800)])
```

Bad items drop: an unknown kind, no known anchor, a box that is not four
finite numbers from 0 to 1000, and a box that is not a list:

``` python
bad = [dict(kind='lasso', box=[0, 0, 10, 10], anchors=['m1']), dict(kind='point', box=[0, 0, 10, 10], anchors=['e7']),
       dict(kind='point', box=[0, 0, float('nan'), 10], anchors=['m1']), dict(kind='point', box=[0, 0, 2000, 10], anchors=['m1']),
       dict(kind='point', box='0,0,10,10', anchors=['m1']), dict(kind='point', box=[5, 0, 5, 10], anchors=['m1']),
       dict(kind='point', box=[0, 0, 10, 10], anchors='m1'), 'circle']
test_eq('focus' in clean_context(dict(anchors=ctx.anchors, focus=bad)), False)
test_eq('focus' in clean_context(dict(anchors=ctx.anchors, focus='m1')), False)
```

At most 5 items stay, and each keeps at most 4 anchors:

``` python
many = [dict(kind='point', box=[0, 0, 10, 10], anchors=['m1', 'passage', 'figure', 'm1'], t=k) for k in range(8)]
c = clean_context(dict(anchors=ctx.anchors, focus=many))
test_eq(([f['t'] for f in c.focus], c.focus[0]['anchors']), ([0, 1, 2, 3, 4], ['m1', 'passage', 'figure']))
```

A good thread stays, the oldest first. A long line is cut to 160
characters, and an unknown tier is the tutor:

``` python
th = [dict(said='I do not get alpha bar', says=['Alpha bar is the share that is left.', 'x' * 200], t='tutor'),
      dict(said='why?', says=['Each step keeps a part of the signal.'], t='magic')]
c = clean_context(dict(anchors=ctx.anchors, thread=th))
test_eq([(t['said'], len(t['says'][1]) if len(t['says']) > 1 else 0, t['t']) for t in c.thread], [('I do not get alpha bar', 160, 'tutor'), ('why?', 0, 'tutor')])
```

A turn keeps what its panel showed, cut to 300 characters. A link in it
drops that part:

``` python
th = [dict(said='why?', says=['Each step keeps a part.'], shown='Panel "Keep a part": rows 1 → 0.9; 2 → 0.81'), dict(said='more', says=['Again.'], shown='see https://x.example')]
test_eq([t.get('shown') for t in clean_context(dict(anchors=ctx.anchors, thread=th)).thread], ['Panel "Keep a part": rows 1 → 0.9; 2 → 0.81', None])
```

A bad thread drops: a thread that is not a list, a turn with no question
or no line, and a line with a tag or a link:

``` python
for bad in ('say it again', dict(said='x'), [dict(said='', says=['a'])], [dict(said='x', says=[])], [dict(said='x', says='a line')],
            [dict(said='x', says=['<script>alert(1)</script>'])], [dict(said='x', says=['see https://example.org'])], [7, None]):
    assert 'thread' not in clean_context(dict(anchors=ctx.anchors, thread=bad)), bad
```

At most the last 3 turns stay, and at most 1500 characters in all. The
oldest turn goes first:

``` python
many = [dict(said=f'question {k}', says=['a line']) for k in range(5)]
test_eq([t['said'] for t in clean_context(dict(thread=many)).thread], ['question 2', 'question 3', 'question 4'])
big = [dict(said=f'q{k}', says=['y' * 160] * 6) for k in range(3)]
th = clean_context(dict(thread=big)).thread
test_eq(([t['said'] for t in th], sum(map(_size, th)) <= THREAD['total']), (['q2'], True))
```

#### The learner’s answer

The page can send the last graded try of the check as the anchor `try`:
the learner’s answer, the outcome, the feedback, the Error line, the
gaps that the answer points at, the state of the check and the tries
that are left. Each field is cut to its bound, an unknown state is
`open`, and a try with no answer and no feedback drops:

``` python
TRY_RAW = dict(id='try', kind='try', text='Alpha bar grows with t.', answer='Alpha bar grows with t, because more noise is added.',
               outcome='Not yet', feedback='Look at what each alpha is: a number below one.', error='Alpha bar does not grow: it is a product of numbers below one.',
               gaps=['cumulative product', 'cumulative product', 'noise schedule'], state='open', left=2)
a = clean_context(dict(anchors=[TRY_RAW])).anchors[0]
test_eq((a['kind'], a['gaps'], a['state'], a['left']), ('try', ['cumulative product', 'noise schedule'], 'open', 2))
a = clean_context(dict(anchors=[dict(TRY_RAW, answer='x' * 900, state='odd', left=True, gaps='not a list')])).anchors[0]
test_eq((len(a['answer']), a['state'], 'left' in a, 'gaps' in a), (TRY['answer'], 'open', False, False))
test_eq(clean_context(dict(anchors=[dict(id='try', kind='try', text='x')])).anchors, [])
```

## The video cues

A page can show a video with its transcript. The transcript is a list of
cues `[start, text]`:
[`video_cues`](https://vedicreader.github.io/rupaka/board.html#video_cues)
makes them from fossick’s cues `{start, end, text}`. A `video_at` line
quotes the transcript at its time (see *The check of one line*).
[`video_id`](https://vedicreader.github.io/rupaka/board.html#video_id)
reads the id of a YouTube link or id, as fossick reads it. It refuses a
link to another site and an id that is not 11 characters.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L645"
target="_blank" style="float:right; font-size:smaller">source</a>

### video_cues

``` python
def video_cues(
    cues
)->list:
```

*fossick’s cues `{start, end, text}` as the board’s `[start, text]`,
with empty lines dropped.*

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L636"
target="_blank" style="float:right; font-size:smaller">source</a>

### video_id

``` python
def video_id(
    url_or_id:str
)->str:
```

*The id of a YouTube link or id, as fossick’s
[`yt_link`](https://vedicreader.github.io/rupaka/board.html#yt_link)
reads it. Raise `ValueError` when it names no video.*

A link, a short link and a bare id give the same id. Another site does
not:

``` python
for u in ('https://www.youtube.com/watch?v=abc123def45&t=3s', 'https://youtu.be/abc123def45', 'abc123def45'): test_eq(video_id(u), 'abc123def45')
test_fail(lambda: video_id('https://example.com/watch?v=abc123def45'))
test_fail(lambda: video_id('https://www.youtube.com/watch?v=short'))
test_eq(video_cues([dict(start=1.234, end=3, text=' Hello  there '), dict(start=4, end=5, text='')]), [[1.23, 'Hello there']])
test_eq(video_id('https://www.youtube.com/shorts/abc123def45'), 'abc123def45')
test_eq(yt_link('abc123def45', 61.5), 'https://www.youtube.com/watch?v=abc123def45&t=61s')
```

### The board as it comes

A board takes some seconds to write. So the host streams it, and the
first step draws while the model writes the next one.
[`stream_board`](https://vedicreader.github.io/rupaka/board.html#stream_board)
reads the reply as it comes. When a new line is in, it reads the whole
lines so far again with
[`parse_board`](https://vedicreader.github.io/rupaka/board.html#parse_board),
so the rules are the same as for a whole reply. When the header says
`sure` of `SURE` or more, `on_step(step, k)` gets each step as soon as
its spoken line and its first mark are in, and again each time it gets
more strokes: so the first mark does not wait for the rest of the step
(the rows of a side panel) or for the next step. The page adds the
strokes that it does not have yet. A step that is not sure, or a reply
with no header, sends nothing: the host can ask again with sources
(abhyasa looks in the vault, then on the web). The board that
[`stream_board`](https://vedicreader.github.io/rupaka/board.html#stream_board)
returns is the truth: its first steps are the steps that went to
`on_step`.

When the stream breaks after a step went out, the board keeps the whole
lines that came, so the page does not lose what it shows.

The learner can stop: a newer question, Esc, or a closed tab. Then the
host sets the `threading.Event` `stop`.
[`stream_board`](https://vedicreader.github.io/rupaka/board.html#stream_board)
reads no more chunks and closes the stream. So the model call ends at
once, and the next question does not wait for the backend.

------------------------------------------------------------------------

<a
href="https://github.com/vedicreader/rupaka/blob/main/rupaka/board.py#L663"
target="_blank" style="float:right; font-size:smaller">source</a>

### stream_board

``` python
def stream_board(
    chunks, ctx, on_step:NoneType=None, sure:float=0.7, stop:NoneType=None, verbatim:function=verbatim
)->fastcore.basics.AttrDict:
```

*[`parse_board`](https://vedicreader.github.io/rupaka/board.html#parse_board)
of the reply that comes in `chunks`. When the header says `sure` or
more, `on_step(step, k)` gets each step as soon as* its first mark is
in, and again each time its strokes change, so the first mark does not
wait for the rows of a panel or for the next step. A panel line with
column names waits for its second row (see
[`_shape`](https://vedicreader.github.io/rupaka/board.html#_shape)). The
page adds the strokes that it does not have yet. When the
`threading.Event` `stop` is set, it reads no more chunks and closes the
stream, so the model call ends and lets go of its backend.
[`verbatim`](https://vedicreader.github.io/rupaka/board.html#verbatim)
checks a quote, as in
[`parse_board`](https://vedicreader.github.io/rupaka/board.html#parse_board).

A sure board of three steps, for the tests:

``` python
three = '\n'.join(json.dumps(o) for o in [dict(op='v', n=1, sure=0.9), dict(op='step', say='Alpha bar t is the share that is left.'),
                                          dict(op='circle', at=dict(a='m1', tex=r'\bar\alpha_t')), dict(op='step', say='Its root scales x0.'),
                                          dict(op='box', at='m1'), dict(op='step', say='The rest is noise.'), dict(op='underline', at='passage')])
```

Each step goes out before the last chunk comes, and the last copy of
each step is the step of the board:

``` python
events, got = [], {}
def chunks(t, n=7):
    for i in range(0, len(t), n): events.append('chunk'); yield t[i:i + n]
b = stream_board(chunks(three), ctx, lambda s, k: (got.__setitem__(k, s), events.append(f'step {k}')))
assert events.index('step 1') < len(events) - 1 - events[::-1].index('chunk'), events
test_eq(([got[k] for k in (1, 2, 3)], b.steps[0]['strokes']), (b.steps, [dict(op='circle', at=dict(a='m1', tex=r'\bar\alpha_t'))]))
```

A board that is not sure sends no step. Its board is the same as the
board of the whole reply:

``` python
got = []
low = three.replace('0.9', '0.4')
test_eq((stream_board(chunks(low), ctx, lambda s, k: got.append(k)).steps, got), (parse_board(low, ctx).steps, []))
```

A broken line in the middle drops as
[`parse_board`](https://vedicreader.github.io/rupaka/board.html#parse_board)
drops it, and the next step still goes out:

``` python
board_rows = three.split('\n')
broken = '\n'.join(board_rows[:3] + ['{"op":"circle","at":"e7"}'] + board_rows[3:])
got = []
last = {}
b = stream_board((broken[i:i + 5] for i in range(0, len(broken), 5)), ctx, lambda s, k: last.__setitem__(k, s))
whole = parse_board(broken, ctx)
test_eq(([last[k] for k in sorted(last)], b.steps, b.dropped), (whole.steps, whole.steps, whole.dropped))
test_eq((len(last), len(b.dropped)), (3, 1))
```

When the stream breaks in step 3, after step 1 went out, the board keeps
the whole lines that came: two steps. A break before any step raises, as
a call does:

``` python
def breaks(text, at):
    yield text[:at]
    raise RuntimeError('the model stopped')
got = []
b = stream_board(breaks(three, three.index('The rest') + 3), ctx, lambda s, k: got.append(k))
test_eq((sorted(set(got)), len(b.steps)), ([1, 2], 2))
test_fail(lambda: stream_board(breaks(three, 5), ctx, lambda s, k: None), contains='the model stopped')
```

A step with a side panel goes out when the panel has its first row, so
the page never shows an empty card, and before the next step. Then it
goes out again as its rows come:

``` python
pan = '\n'.join(json.dumps(o) for o in [dict(op='v', n=1, sure=0.9), dict(op='step', say='Each test is a question: 1.0 gives 1.414, and 2.0 gives 2.0.'), dict(op='panel', head='Make a question'),
    dict(op='row', cells=['[1.0]'], to='1.414', at='passage'), dict(op='row', cells=['[2.0]'], to='2.0'), dict(op='step', say='Then it adds up.')]) + '\n'
seen_ = []
stream_board((pan[i:i + 4] for i in range(0, len(pan), 4)), ctx, lambda s, k: seen_.append((k, len(s['strokes']))))
test_eq(seen_, [(1, 2), (1, 3), (2, 0)])
```

A panel line with column names waits for its second row, because the
first two rows decide whether the names stay:

``` python
held = '\n'.join(json.dumps(o) for o in table[:5]) + '\n'
seen_ = []
stream_board((held[i:i + 4] for i in range(0, len(held), 4)), actx, lambda s, k: seen_.append([x['op'] for x in s['strokes']]), sure=0)
test_eq(seen_, [[], ['panel', 'row'], ['row']])   # step 1 goes out with no panel when step 2 comes; its panel and row come with the second row
seen_ = []
b = stream_board((held[i:i + 4] for i in range(0, len(held) - 60, 4)), actx, lambda s, k: seen_.append([x['op'] for x in s['strokes']]), sure=0)
test_eq((seen_, [s['op'] for s in b.steps[0]['strokes']]), ([[], ['panel', 'row'], []], ['panel', 'row']))   # a reply that ends before row 2: the panel goes out at the end
```

The stream keeps the pairing: the row of the screenshot does not go out
with step 1. It goes out with step 2, whose line names it, and step 1
never gets it:

``` python
sent = []
neuron_r = '\n'.join(json.dumps(o) for o in neuron[:7]) + '\n'
stream_board((neuron_r[i:i + 9] for i in range(0, len(neuron_r), 9)), nctx, lambda s, k: sent.append((k, [x['op'] for x in s['strokes']])))
test_eq(any(k == 1 and 'row' in ops for k, ops in sent), False)
test_eq(sent[-1], (2, ['row', 'row']))
```

A set `stop` ends the read at the next chunk, and the stream closes. No
more step goes out:

``` python
import threading
```

``` python
stop, got, shut = threading.Event(), [], []
def slow():
    try:
        for i in range(0, len(three), 7):
            if i > three.index('Its root') + 30: stop.set()   # the learner stops while step 2 comes
            yield three[i:i + 7]
    finally: shut.append(True)
b = stream_board(slow(), ctx, lambda s, k: got.append(k), stop=stop)
test_eq((got, shut), ([1], [True]))
```

A host gives its own check of a quote to the stream too, and the stream
gives it to each read:

``` python
said = []
stream_board(chunks(json.dumps(good)), qctx, verbatim=lambda q, t: said.append(q) or '')
assert said
```