spec

Build a demo spec in Python, check it, and move an old spec to the new schema.

A demo is one JSON object: the spec. It lists blocks, a small state, derived values and data tables. This module reads the block and fn manifests, gives a builder for each block, and checks a spec with validate.

validate enforces the think rule: a block that shows a value which a gate’s answer depends on must wait for that gate. So the learner always predicts before the demo shows the result.

Manifests

Each block is a folder in rupaka/blocks/, with a block.json manifest. Each fn has a manifest in rupaka/fns/. A manifest gives the purpose in one line, the props with their types, and the keys. The validator and the agent prompt read the manifests, so they never drift from the code.

source

fn_manifests

def fn_manifests()->dict:

The fn manifests, by name.

source

manifests

def manifests()->dict:

The block manifests, by name.

Each block folder has its code, its style, its manifest and one example.

test_eq(set(manifests()), {'axis', 'bars', 'caption', 'choice', 'gate', 'grid', 'plot', 'point', 'ranklist', 'readout', 'row', 'slider', 'stack', 'toggle', 'wheel'})
for n in manifests(): assert all((PKG/'blocks'/n/f).exists() for f in (f'{n}.js', f'{n}.css', 'example.json')), n

The fns are rules, rrf, softmax, lookup, expr, and four small helpers.

test_eq(set(fn_manifests()), {'expr', 'fmt', 'ids', 'lookup', 'pick', 'rrf', 'rules', 'softmax', 'sort'})

Builders

A builder makes the dict for one block, so a notebook author does not type JSON. A prop with the value None is left out.

source

demo

def demo(
    id:str, title:str='', ask:str='', state:dict=None, derive:tuple=(), blocks:tuple=(), tables:dict=None,
    concept:str=None
)->dict:

A whole demo spec.

source

is_

def is_(
    v
)->dict:

A fixed gate answer.

source

from_

def from_(
    path:str
)->dict:

Gate options read from a list in the state, for example from_('join.alts').

These are the builders for the blocks. Each one takes the props of its manifest.

source

grid

def grid(
    bind, label:NoneType=None, **kw
):

source

plot

def plot(
    series, **kw
):

source

point

def point(
    bind:NoneType=None, **kw
):

source

axis

def axis(
    x, *blocks, y:NoneType=None, **kw
):

source

bars

def bars(
    bind, label:NoneType=None, **kw
):

source

ranklist

def ranklist(
    bind, label:NoneType=None, **kw
):

source

wheel

def wheel(
    bind, items, label:NoneType=None, **kw
):

source

choice

def choice(
    bind, options, label:NoneType=None, **kw
):

source

toggle

def toggle(
    bind, label:NoneType=None, **kw
):

source

slider

def slider(
    bind, label:NoneType=None, min:NoneType=None, max:NoneType=None, step:NoneType=None, **kw
):

source

stack

def stack(
    *blocks, **kw
):

source

row

def row(
    *blocks, **kw
):

source

caption

def caption(
    text, label:NoneType=None, **kw
):

source

readout

def readout(
    text, label:NoneType=None, **kw
):

source

gate

def gate(
    id, prompt, options, answer, why:NoneType=None, context:NoneType=None, again:NoneType=None, hides:NoneType=None
):

Predict, then reveal: blocks with after=id wait for this gate.

The derive builders name a fn and the key it writes (out).

source

fmt

def fmt(
    text, out
):

source

expr

def expr(
    f, out, over:NoneType=None
):

source

lookup

def lookup(
    table, key, out, by:NoneType=None
):

source

softmax

def softmax(
    src, out, t:NoneType=None, labels:NoneType=None, sort:NoneType=None
):

source

rrf

def rrf(
    ins, k, out
):

source

rules

def rules(
    table, ins, out, alts:int=4
):

A small demo: two numbers, a gate on their sum, and a readout that waits for the gate.

d = demo('sum', 'Sum', state=dict(a=2, b=3), derive=[expr('a + b', 'sum')],
         blocks=[gate('s', 'What is {a} + {b}?', [4, 5, 6], 'sum'), readout('{a} + {b} = {sum}', after='s')])
test_eq(d['blocks'][1], {'type': 'readout', 'text': '{a} + {b} = {sum}', 'after': 's'})
test_eq(d['derive'], [{'fn': 'expr', 'expr': 'a + b', 'out': 'sum'}])

The expr grammar

expr holds a formula in a tiny safe grammar: numbers, names, + - * / ^, parentheses and a few math functions. The runtime parses it in JavaScript. This is the same parser in Python, so validate can check a formula and run it. There is no eval in either one. A name must be a state key, a constant or one of the functions.

_safe turns a math error into the value JavaScript gives, so both sides agree: nan or an infinity.

parse_expr returns a function of a dict of values, and the set of names that the formula reads.

source

parse_expr

def parse_expr(
    src:str
):

Parse a formula. Return (f, names): f(env) gives a number, and names are the values it reads.

2*x^2 - 1 at x = 3 is 17. The power binds before the minus sign, so -x^2 at x = 3 is −9.

f, ns = parse_expr('2*x^2 - 1')
test_eq(f(dict(x=3)), 17)
test_eq(ns, {'x'})
test_eq(parse_expr('-x^2')[0](dict(x=3)), -9)
test_eq(parse_expr('max(a, b) + floor(7/2)')[0](dict(a=1, b=4)), 7)

Names that reach into objects, calls of unknown functions, and a second statement are refused.

for bad in ('constructor', '__proto__', 'alert(1)', 'x;y', 'x.y', '2 +'): test_fail(lambda: parse_expr(bad)[0](dict(x=1, y=2)), contains='expr')

The fns in Python

Each fn has a Python twin with the same result. validate runs the derive list on the opening state with these twins, so it can check that each gate answer is one of the options. The browser tests compare the two sides.

First, two helpers that copy the runtime: get reads a path such as fused.0.id, and show_value writes a value as the runtime does.

source

fmt_text

def fmt_text(
    s, st
)->str:

Fill {path} and {path:2} from the state, as the runtime does.

source

show_value

def show_value(
    v, d:NoneType=None
)->str:

A value as the runtime shows it: numbers with at most d (default 4) digits and a true minus sign.

source

js_str

def js_str(
    v
)->str:

String(v) in JavaScript, for the values a spec holds.

source

get

def get(
    o, path
):

Read a path such as fused.0.id in dicts and lists; own keys only, as the runtime does.

st = dict(fused=[dict(id='B', value=0.0328)], k=-60, xs=[1, 2])
test_eq(get(st, 'fused.0.id'), 'B')
test_eq(get(st, 'fused.0.__proto__'), None)
test_eq(fmt_text('{fused.0.id} {fused.0.value:2} {k} {xs}', st), 'B 0.03 −60 1, 2')

Now the fns. rules orders its alts with the same small hash as the runtime, so both sides give the same options.

rrf, softmax and lookup follow the runtime line by line.

expr gives a number, or a series of points with over. The small helpers finish the set.

run gives the opening state of a demo: the state, then each derive in order. Each gate starts open.

source

run

def run(
    spec, state:dict=None
)->dict:

The state of spec after the derive list, from its opening state or from state.

The fused scores of the two spike lists at k = 60: B is first, because it is in both lists.

sp = dict(state=dict(bm25=list('ACEB'), vec=list('DFGB'), k=60), derive=[rrf(['bm25', 'vec'], 'k', 'fused')], blocks=[])
st = run(sp)
test_eq(st['fused'][0]['id'], 'B')
test_close(st['fused'][0]['value'], 1/64 + 1/64)
test_eq(run(sp, {**sp['state'], 'k': 1})['fused'][0]['id'], 'A')

softmax at a high temperature makes the shares more equal.

sp = dict(state=dict(z=[2, 1, 0]), derive=[softmax('z', 'p', t=1), softmax('z', 'q', t=2)], blocks=[])
st = run(sp)
test_close(st['p'][0]['value'], 0.6652, eps=1e-4)
assert st['q'][0]['value'] < st['p'][0]['value']

validate

validate(spec) returns a list of errors. Each error has a path and one sentence, for example blocks[3].bind: "fused" is not in state or derive. An empty list means the spec is good. It checks, in order:

  1. The shape: the schema number, the top-level keys, and a list of blocks.
  2. Each derive: the fn exists, its props have the right types, and it reads only keys that come before it.
  3. Each block: the type exists, its props have the right types, the required props are there, and each bind names a key. Each after and each lock names a gate, and no gate waits for itself. A block above its gate can wait for it, for example a mark on a figure.
  4. Each gate: its answer resolves on the opening state and is one of its options. A gate whose answer a live control can change names its inputs with context.
  5. The think rule (below).
  6. Text: each sentence has at most 20 words.

First, the prop types. _reads gives the keys that a value of a type reads, so the think rule can follow them.

_props checks the props of one block or one derive entry against its manifest. It returns the keys that the entry reads.

_walk visits every block with its path and its parent, in reading order.

The think rule

The answer of a gate has a root key: for fused.0.id, it is fused. A block must wait for the gate when it reads a key that gives the answer away:

  • the root key itself;
  • each derived key that the root is computed from (raw inputs are the question, so they stay visible);
  • each derived key that is computed from any of these.

A fixed answer ({"is": ...}) has no root key, so the gate names the keys that give it away in hides. A block waits for the gate when it, or a block that holds it, has after set to that gate, or to a gate that itself waits for it.

Text

ASD-STE100 asks for sentences of at most 20 words. This cheap check counts the words in each sentence of each text prop.

source

long_sentences

def long_sentences(
    s:str
)->list:

The sentences of s that have more than MAX_WORDS words.

test_eq(long_sentences('Short one. ' + 'word ' * 21 + 'end.'), ['word ' * 21 + 'end.'])

validate

Now validate puts the checks together.

source

validate

def validate(
    spec
)->fastcore.foundation.L:

The errors of spec, each as path: sentence. An empty list means the spec is good.

_check_blocks does the block checks, the gate checks and the think rule.

_check_gate checks one gate: its answer, its options, its context, and the think rule for its answer.

Tests for validate

A good demo has no errors.

good = demo('sum', 'Sum', state=dict(a=2, b=3), derive=[expr('a + b', 'sum')],
            blocks=[gate('s', 'What is {a} + {b}?', [4, 5, 6], 'sum'), readout('{a} + {b} = {sum}', after='s')])
test_eq(validate(good), [])

A readout of the sum before the gate fails the think rule, with the path of the readout.

bad = deepcopy(good); del bad['blocks'][1]['after']
test_eq(validate(bad), ['blocks[1]: this readout shows "sum", which gives away the answer of gate "s"; add "after": "s"'])

An unknown block, an unknown fn, an unknown prop and a missing prop each fail with a path.

def errs_of(**kw): return validate({**deepcopy(good), **kw})
test_eq(errs_of(blocks=[*good['blocks'], dict(type='dial')])[0][:23], 'blocks[2].type: "dial" ')
assert errs_of(derive=[dict(fn='sum', out='sum')])[0].startswith('derive[0].fn: "sum" is not a fn')
assert errs_of(blocks=[good['blocks'][0], {**good['blocks'][1], 'colour': 'red'}])[0].startswith('blocks[1].colour: is not a prop of readout')
assert 'blocks[1]: needs "text"' in validate({**good, 'blocks': [good['blocks'][0], dict(type='readout', after='s')]})[0]

A bind to a key that is not in the state fails. So does an after that names no gate.

assert 'blocks[2].bind: "k" is not in state or derive' in errs_of(blocks=[*good['blocks'], slider('k')])
test_eq(errs_of(blocks=[readout('{sum}', after='t'), good['blocks'][0]]), ['blocks[0].after: "t" is not a gate', 'blocks[0]: this readout shows "sum", which gives away the answer of gate "s"; add "after": "s"'])

A gate answer must be one of its options. A demo needs a gate.

test_eq(errs_of(blocks=[gate('s', 'Sum?', [1, 2], 'sum')]), ['blocks[0].answer: the answer "5" is not one of the options [\'1\', \'2\']'])
test_eq(errs_of(blocks=[readout('{a}')]), ['blocks: needs at least one gate: the learner predicts first'])

A live control that changes the answer needs a context on the gate, so the verdict names its inputs.

live = errs_of(blocks=[slider('a', lock='s'), *good['blocks']])
test_eq(live, ['blocks[1].context: the slider at blocks[0] changes the answer, so name the inputs, for example "For {a}:"'])
ok = deepcopy(good); ok['blocks'] = [slider('a', lock='s'), {**good['blocks'][0], 'context': 'For {a} + {b}:'}, good['blocks'][1]]
test_eq(validate(ok), [])

The think rule follows derived keys both ways. Here twice is computed from the answer sum, so a readout of twice must wait too.

t = deepcopy(good); t['derive'].append(expr('2 * sum', 'twice')); t['blocks'].append(readout('{twice}'))
test_eq(validate(t), ['blocks[2]: this readout shows "twice", which gives away the answer of gate "s"; add "after": "s"'])

A block inside a stack that waits for the gate passes, because the stack waits.

t['blocks'][2] = stack(readout('{twice}'), after='s')
test_eq(validate(t), [])

A fixed answer names the keys that give it away in hides.

h = demo('h', state=dict(m=1), derive=[expr('m * x', 'line', over=dict(x=[-1, 1, 3]))],
         blocks=[gate('g', 'Steeper or flatter?', ['steeper', 'flatter'], is_('steeper'), hides=['line']),
                 axis([-1, 1], plot('line'), y=[-3, 3])])
test_eq(validate(h), ['blocks[1].blocks[0]: this plot shows "line", which gives away the answer of gate "g"; add "after": "g"'])

A plain-text prop, such as ask or a label, cannot hold {path}: the runtime would show the braces.

assert errs_of(ask='The list has {a} items.')[0].startswith('ask: is plain text')
assert errs_of(blocks=[*good['blocks'], readout('{a}', label='a is {a}')])[0].startswith('blocks[2].label: is plain text')

Each sentence has at most 20 words.

assert 'more than 20 words' in errs_of(ask=' '.join(['word'] * 25) + '.')[0]

upgrade

upgrade(spec) moves a spec from schema N−1 to N. Schema 1 is the first, so today the step is the identity. The frame and the test are in place for schema 2.

source

upgrade

def upgrade(
    spec:dict
)->dict:

A copy of spec at the current schema.

test_eq(upgrade({**good, 'rupaka': 0}), good)
test_fail(lambda: upgrade({**good, 'rupaka': 9}), contains='newer')

The block examples

Each block’s example.json must pass validate. The browser tests draw each one.

source

examples

def examples()->dict:

The example spec of each block, by block name.

for n, ex in examples().items(): test_eq((n, validate(ex)), (n, []))