web search, fetch, crawl, and browser automation for Python and AI agents
search finds sources. read turns a supported source into markdown with one result shape. fetch and to_md handle web pages, from plain HTTP through browser-backed requests. cdp_connect drives Chrome for authenticated or multi-step work. Dedicated readers handle YouTube, arXiv, GitHub, and PDFs.
Install
uv add fossick
Text/image/news search need no Docker. JS rendering, stealth fetch, and google() use a bundled headless browser.
Quick start
results = search('koshas github fts semantic code graph', method='flashrank', n=5)for r in results: print(r['score'], r['engines'], r['title'], r['href'])
res = research('sqlite WAL mode vs journal mode', n=5)print(res['digest'][:500])print([s['href'] for s in res['sources']])
## Write-Ahead Logging
https://www.sqlite.org/wal.html
Write-Ahead Logging
Table Of Contents
# 1\. Overview
The default method by which SQLite implements atomic commit and rollback is a rollback journal. Beginning with version 3.7.0 (2010-07-21), a new "Write-Ahead Log" option (hereafter referred to as "WAL") is available.
There are advantages and disadvantages to using WAL instead of a rollback journal. Advantages include:
1. WAL is significantly faster in most scenarios.
2. WAL provid
['https://www.sqlite.org/wal.html', 'https://blog.sqlite.ai/journal-modes-in-sqlite', 'https://mohit-bhalla.medium.com/understanding-wal-mode-in-sqlite-boosting-performance-in-sql-crud-operations-for-ios-5a8bd8be93d2', 'https://til.simonwillison.net/sqlite/enabling-wal-mode', 'https://fly.io/blog/wal-mode-in-litefs/']
The legality of web scraping varies across the world. In general, web scraping may be against the terms of service of some websites, but the enforceability of these terms is unclear.[11]
In the United States, website owners can use three major legal claims to prevent undesired web scraping: (1) copyright infringement (compilation), (2) violation of the Computer Fraud and Abuse Act ("CFAA"), and (
One door
what_is classifies a target as dir, file, arxiv, youtube, github, ghfile, pdf, or web. read selects that reader and returns ok, kind, title, source, text, skipped, and meta.
Directories and GitHub repositories are trees. Their result has a local path in meta and no text. Other successful readers return markdown in text. For PDFs, pages=True returns (page_number, text) pairs for page-level citations.
A recognized target can still fail to yield content. Bot walls, missing YouTube transcripts, and invalid PDF responses set ok=False and put the reason in skipped. Unsupported targets raise ValueError during classification.