stack

which of your own packages is installed, where from, and which one just raised

When a release fails, the traceback often runs through several of your own packages, sometimes through installed copies of them rather than the checkouts you edit. This module answers three questions: which of your packages are installed, where an import of each one resolves to, and which one raised an error.

Nothing here imports the packages it reports on. Every answer comes from import metadata, a file path, or the text of a traceback, so a package that fails to import can still be described.

FAMILY is the default list of your own packages: the ones pullup is built from, and the only ones a report blames. A frame in json or asyncio is someone else’s code and is passed over. Every function that takes family= uses your list instead, and an empty family means FAMILY.

source

installed

def installed(
    name
):

Whether name is importable, without importing it.

installed says whether Python can find the module, without importing it. A module that is found but fails when imported still counts as installed.

installed('nbdev'), installed('no_such_package_here'), installed('')
(True, False, False)

source

checkout_for

def checkout_for(
    name, checkouts
):

The folder in checkouts that holds package name, found by its __init__.py rather than the folder name.

source

survey

def survey(
    family:tuple=(), checkouts:tuple=()
):

One row per family package: installed, version, the copy an import resolves to, and whether a checkout is shadowed.

survey describes each package in the family in one row: whether it is installed, its version, and the path an import resolves to. source is checkout when that path is a working tree and site-packages when it is inside an installed environment, and editable is true in the first case. Pass the folders you work in as checkouts, and checkout names the one that holds the package. shadowed is true when you have a checkout of a package but Python imports an installed copy instead, so your edits have no effect:

[{k: r[k] for k in ('name', 'installed', 'source', 'shadowed')}
 for r in survey(family=['pullup', 'nbdev', 'dhrishti'])]
[{'name': 'pullup',
  'installed': True,
  'source': 'checkout',
  'shadowed': False},
 {'name': 'nbdev',
  'installed': True,
  'source': 'site-packages',
  'shadowed': False},
 {'name': 'dhrishti', 'installed': False, 'source': '', 'shadowed': False}]

The checkout is matched by what is inside it rather than by its folder name, so a fork called nbdev-fork still holds the nbdev package and is still the checkout for it.

fork = root/'nbdev-fork'; (fork/'nbdev').mkdir(parents=True)
(fork/'nbdev'/'__init__.py').write_text('')
row = survey(family=['nbdev'], checkouts=[fork])[0]
{k: row[k] for k in ('source', 'editable', 'shadowed')}
{'source': 'site-packages', 'editable': False, 'shadowed': True}

source

frames

def frames(
    text
):

Every File ..., line N, in f in a traceback, oldest first, as the terminal printed it.

frames pulls every File "...", line N, in f line out of a traceback’s text, oldest first, as the terminal printed them. Text without frames gives an empty list. The example builds a small package that raises:

pkg = root/'blowup'; pkg.mkdir()
(pkg/'__init__.py').write_text('')
(pkg/'core.py').write_text("""def push(remote):
    raise RuntimeError(f"no upstream for {remote}")

def sync():
    push("origin")
""")
sys.path.insert(0, str(root))
from blowup.core import sync
try: sync()
except RuntimeError: tb = traceback.format_exc()
print(tb.replace(str(root), '/proj'))
Traceback (most recent call last):
  File "<ipython-input-23-1bc83a927e6d>", line 2, in <module>
    try: sync()
         ^^^^^^
  File "/proj/blowup/core.py", line 5, in sync
    push("origin")
  File "/proj/blowup/core.py", line 2, in push
    raise RuntimeError(f"no upstream for {remote}")
RuntimeError: no upstream for origin

The deepest frame is the one that ran the raise, and it is the last one frames returns.

{**frames(tb)[-1], 'file': frames(tb)[-1]['file'].replace(str(root), '/proj')}
{'file': '/proj/blowup/core.py', 'line': 2, 'fn': 'push'}

_package_of names the family package a file belongs to. For a file under site-packages it is the folder directly below site-packages, so a copy vendored inside another package is not blamed. For any other path it is the last folder whose name is in the family:

(_package_of('/home/me/code/gheasy/gheasy/repo.py', ['gheasy']),
 _package_of('/x/.venv/lib/python3.11/site-packages/vendorpkg/gheasy/repo.py', ['gheasy']),
 _package_of('/usr/lib/python3.11/json/decoder.py', ['gheasy']))
('gheasy', '', '')

_is_error picks out the line that names the exception, such as RuntimeError: no upstream for origin or a bare RuntimeError, and rejects the header, the frame lines and echoed source lines. blame reads the traceback from the bottom and takes the first line it accepts:

[_is_error(l) for l in ('Traceback (most recent call last):',
                        '  File "core.py", line 2, in push',
                        '    if not upstream:',
                        'RuntimeError',
                        'RuntimeError: no upstream for origin')]
[False, False, False, True, True]

source

blame

def blame(
    text, checkouts:tuple=(), family:NoneType=None
):

Which family package raised, from the deepest family frame, and the file to open, preferring your checkout.

blame puts these together. It finds the deepest frame that belongs to a family package, takes the error line, and returns the package, file, line, function, error, installed version and checkout. open is the file to open in an editor. When no frame belongs to the family, package is empty but error is still filled in:

r = blame(tb, checkouts=[root], family=['blowup'])
{k: (v.replace(str(root), '/proj') if isinstance(v, str) else v) for k, v in r.items()}
{'package': 'blowup',
 'file': '/proj/blowup/core.py',
 'line': 2,
 'fn': 'push',
 'error': 'RuntimeError: no upstream for origin',
 'open': '/proj/blowup/core.py',
 'checkout': '/proj',
 'version': ''}

The traceback names the copy that ran. When you pass checkouts and that copy was installed rather than checked out, open points at the same file in your checkout, which is the copy you can edit:

other = root/'blowup-fork'; (other/'blowup').mkdir(parents=True)
(other/'blowup'/'__init__.py').write_text(''); (other/'blowup'/'core.py').write_text('')
moved = blame(tb, checkouts=[other], family=['blowup'])
moved['file'] == moved['open'], moved['open'].startswith(str(other))
(False, True)

An exception raised with no message still has an error line, and it is the class name alone.

In a chained traceback, error is the last exception raised and the frame is the deepest family frame anywhere in the text. Those two are not always parts of the same exception.