hl

Syntax colouring for code and for unified diffs, in the terminal or as HTML.

Code

hl_text(code, lang, theme) colours a piece of code for the terminal. theme is a Lumis theme name, and HL_THEMES gives the Lumis theme for each palette name the terminal accepts, so --theme nord colours code with nord. scope_style looks up the style a theme gives one scope. A scope the theme does not name uses its longest dotted prefix that it does name, so function.builtin falls back to function.

---

source

hl_text

def hl_text(
    code, lang, theme:str='github_dark'
):

code as a rich Text, coloured by fastpylight with Lumis theme.

---

source

scope_style

def scope_style(
    scope, theme:str='github_dark'
):

The rich style Lumis theme gives token scope; the longest dotted prefix it names wins.

fastpylight counts positions in UTF-8 bytes and rich counts characters, and hl_text converts between them, so each coloured span starts on the character its token starts on. A line ending that rich rewrites, such as \r\n, leaves the spans in place:

t = hl_text('a\r\nb = 1  # one', 'python')
test_eq(t.plain, 'a\nb = 1  # one')
assert any(t.plain[s.start:s.end] == '1' for s in t.spans)
t

hl_toks(code, lang) returns the tokens as data instead: one list per line, each token a (start, end, scope) tuple in characters. Use it when you draw the code yourself. A token that runs across a line break, such as a triple-quoted string, is cut at the break and appears on both lines:

---

source

hl_toks

def hl_toks(
    code, lang
):

Per line of code, its tokens as (start, end, scope) in characters; a token crossing a line break is cut at it.

t = hl_toks('é = """a\nb"""', 'python')
test_eq(t[0][0], (0, 1, 'variable'))
test_eq([k for *_, k in t[1]], ['string'])

Diffs

diff_rich(s, palette, theme) colours a unified diff for the terminal. It also colours a diffstat, the file | 3 ++- summary git prints, and returns any other text in gray. palette maps fg1, gray, red, green, aqua and bg0 to colours. File headers are bold, hunk headers are aqua, and added and removed lines get a background tinted 15% toward green or red. The code inside each hunk is coloured with theme in its own language. The old side and the new side of a hunk are each tokenized whole, so a string that spans several lines keeps its colour on every one. is_diff says whether a text holds a unified diff at all.

The same split is available as data. diff_toks returns one entry per line of a diff: the line’s tokens, or None for a line outside a hunk body, such as a header. hunk_toks does the same for one hunk body. diff_html renders each code line as HTML through toks_html, which wraps each token in a span whose classes are its scope and every prefix of it, as in hl-function hl-function-builtin, so a stylesheet that styles only function still colours it.

diff_lang picks the language of a hunk from the path on its --- or +++ line. A notebook’s diff holds the source of its cells, so a path ending in .ipynb reads as Python. So does a single cell, written nb.ipynb#id the way edit_cell writes it. Any other path takes fastpylight’s guess from the file name. The cells below check diff_lang, then diff_html on a small diff.

---

source

diff_rich

def diff_rich(
    s, palette, theme:str='github_dark'
):

s coloured in palette when it holds a unified diff, its code by the file’s language, or a diffstat; else gray.

---

source

diff_html

def diff_html(
    s, lang:NoneType=None
):

Per line of unified diff s, the HTML of its code after the prefix, or None outside a hunk body.

---

source

toks_html

def toks_html(
    code, toks
):

code escaped, each token a span whose classes are its scope and every dotted prefix of it, as fastpylight.theme_css names them.

---

source

diff_toks

def diff_toks(
    s, lang:NoneType=None
):

Tokens per line of unified diff s, relative to the line after its prefix; None for every line outside a hunk body. lang overrides the ---/+++ paths.

---

source

diff_lang

def diff_lang(
    path
):

fastpylight’s language for a diffed path; a notebook, or one cell of it as nb.ipynb#id, is Python.

---

source

is_diff

def is_diff(
    s
):

Whether s holds a unified diff: at least one hunk header.

---

source

hunk_toks

def hunk_toks(
    ls, lang
):

Tokens per hunk body line, relative to the line after its prefix; each side is tokenized whole, so a string spanning lines keeps its scope. None for a \ line.

test_eq([diff_lang(p) for p in ('a/x.py', 'b/nbs/05_cli.ipynb', 'b/nbs/05_cli.ipynb#2874a64d', 'a/README.md')],
        ['python', 'python', 'python', 'markdown'])

d = '--- a/x.py\n+++ b/x.py\n@@ -1,3 +1,4 @@\n import os\n-x = "a<b"\n+s = """one\n+two"""\n é = 1  # ü\n\\ No newline at end of file'
h = diff_html(d)
test_eq(len(h), len(d.split('\n')))
test_eq(h[:3] + h[-1:], [None] * 4)
assert '&lt;' in h[4] and 'hl-string' in h[6]
assert '# ü' in h[7] and h[7].startswith('<span class="hl-variable">é</span>')
test_eq(diff_html('@@ -1 +1 @@\n-a<\n+b', 'plaintext'), [None, 'a&lt;', 'b'])
test_eq(toks_html('len(x)', [(0, 3, 'function.builtin')]), '<span class="hl-function hl-function-builtin">len</span>(x)')

Here is a notebook cell’s diff, as edit_cell reports it, in the default github-dark colours. The added line carries the colour of its + mark, the green tint, and the code’s own colours:

dark = {'fg1': '#c9d1d9', 'gray': '#8b949e', 'red': '#ff7b72', 'green': '#3fb950', 'aqua': '#39c5cf', 'bg0': '#0a0c10'}
nb_diff = diff_rich('--- a/nbs/x.ipynb#c1\n+++ b/nbs/x.ipynb#c1\n@@ -1 +1 @@\n-x = 1\n+def f(): return 2', dark)
added = nb_diff.split('\n')[-1]
test_eq(added.plain, '+def f(): return 2')
assert len({str(s.style) for s in added.spans}) > 2
test_eq(diff_rich('3 passed', dark).spans, [])
nb_diff