search

Web search across several engines, merging and reranking the results, and research digests

search asks several search engines the same question at once and merges their answers into one list. Each engine misses pages the others find, and a page that several engines rank highly is usually a good one. fossick reaches the engines through ddgs, which needs no API keys. It merges the lists with reciprocal rank fusion (RRF), explained below. It can then reorder the merged list by how well each result matches the query.

google gets Google’s own ranking by loading the results page in a stealth browser. It is slower than search. research searches, downloads the best results that have readable text, and returns them as one markdown digest that cites each source. You can pass the digest to a language model as it is.

The ddgs client

get_ddgs returns a shared DDGS client for each timeout and proxy setting, created on first use. ddgs_env reads proxy and certificate settings from the environment. This lets search work behind a corporate proxy. It reads the proxy from DDGS_PROXY, HTTPS_PROXY or https_proxy. It reads a certificate bundle from FOSSICK_CA_BUNDLE, SSL_CERT_FILE, REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE.

get_ddgs

def get_ddgs(
    timeout:int=10
):

Return the shared ddgs client for this timeout and proxy and certificate settings, creating it on first use.

ddgs_env

def ddgs_env()->dict:

Return proxy and certificate settings for ddgs, read from the environment.

Asking each engine

For text search, fossick queries each engine in TEXT_BACKENDS separately and keeps each engine’s results in the order that engine ranked them. It does not use ddgs’s own combined search, which picks engines at random and favours Wikipedia. backends lists every engine ddgs offers for a category. With pages above 1, fossick asks each engine for that many pages of results, and drops results on later pages that repeat earlier ones. The test below checks that the ranks keep counting across pages.

backends

def backends(
    category:str='text'
)->list:

Return the names of every ddgs engine available for category.

Merging the lists

Reciprocal rank fusion (RRF) merges ranked lists without needing the engines’ own scores, which are not comparable. Each page gets 1/(k + rank) from every list it appears in, and those amounts are added up. A page ranked first by one engine scores more than a page ranked eighth, and a page found by several engines adds up a score from each. k, 60 by default, controls how much more the top ranks count than the lower ones.

rrf treats two URLs as the same page when they differ only in scheme, www., a trailing slash or tracking parameters, using norm_url from fossick.quality. For each page it keeps the longest snippet any engine gave, the fused score, and engines, the list of engines that found it.

rrf

def rrf(
    lists:list, # ranked result lists, one per backend
    k:int=60, # RRF damping constant
)->list:

Merge ranked lists with reciprocal rank fusion, scoring each URL by the sum of 1/(k+rank) over the lists it appears in.

Reranking

Search engines rank by many things besides relevance to the exact words asked. rerank reorders results by how well their title, snippet and URL match the query. It blends that score with the merged score. mix is the weight of the merged score. The default is 0.4. At 1 the reranker is ignored. When method is not 'none', rerank adds a rerank field with the relevance score to each result.

method chooses the scorer. 'lexical' uses BM25, a standard word-matching formula. bm25 computes scores from the result list. It needs no stored index. 'flashrank' uses the small cross-encoder model of flashrank. The model reads the query and each result together and matches meaning better. It needs pip install flashrank. flashrank_scores returns None when flashrank is missing, and method='flashrank' then raises ImportError. 'auto' uses flashrank when installed and BM25 otherwise. 'none', the default for search, keeps the merged order.

rerank

def rerank(
    q:str, # search query
    hits:list, # fused hits (from `rrf`) or any list of result dicts
    n:int=None, # keep the top n (None = all)
    method:str='none', # none | lexical | flashrank | auto
    mix:float=0.4, # weight on the retrieval score vs the reranker (1 = ignore the reranker)
)->list:

Reorder hits by relevance to q, blending the reranker’s score with the merged search score.

flashrank_scores

def flashrank_scores(
    q:str, texts:list, model:str=None, cache_dir:str=None
)->list | None:

Return flashrank cross-encoder scores for texts against q, or None when flashrank is not installed.

bm25

def bm25(
    q:str, docs:list, k1:float=1.5, b:float=0.75
)->list:

Return the BM25 score of each of docs for the query q, with word weights computed from docs themselves.

Where and when

The best answer to many questions depends on the country. Building rules, prices and what is for sale all differ. Search engines take a region code of the form country-language, such as au-en. With region='auto', the default, fossick works out the two halves separately from the query.

infer_country looks for country names, adjectives and big cities from COUNTRY_WORDS. It answers only when the query names exactly one country. A query that names two countries gets None. infer_language counts common function words from LANG_WORDS, such as “the” and “what” or “der” and “und”. It answers only when one language has the most matches and has at least two. infer_region joins the two results. A query that names no language takes the usual language of the country it names, from COUNTRY_LANG. The default us-en fills whatever the query leaves open. An English question about France gives fr-en.

timelimit keeps only recent results: 'd', 'w', 'm' or 'y' for the past day, week, month or year.

infer_region

def infer_region(
    q:str, # the query, read for a place and a language
    dflt:str='us-en', # used for whichever half the query does not settle
)->str:

Return a ddgs country-language region read from q, filling the parts it leaves open from dflt.

infer_language

def infer_language(
    q:str
)->str:

Return the language code that the function words in q point to, or None when it is unclear.

infer_country

def infer_country(
    q:str
)->str:

Return the country code of the one country named in q, or None when it names none or several.

`search`

search is the main entry point. For text it queries every engine in TEXT_BACKENDS in parallel, merges the lists with RRF and reranks them with method. backend='google' restricts it to one engine. n only cuts the final list and does not change how many engines are asked. Raise pages to get more candidates. With fuse=False, or for the images, news and books categories, it uses ddgs’s own search for that category. The videos category searches YouTube with search_yt.

An empty query returns an empty list without touching the network. When every engine fails, search prints one warning per process and returns an empty list. text, images, news, videos and books are search with the category set. extract downloads one URL with ddgs and returns {url, content}, with the content as markdown by default.

extract

def extract(
    url, **kw
):

Download url with ddgs and return {url, content}, the content as markdown by default.

search

def search(
    q:str, # search query
    category:str='text', # text | images | news | videos | books
    n:int=10, # number of results to return
    fuse:bool=True, # fan out across backends and fuse with RRF (text only)
    backend:str=None, # restrict to a single ddgs backend, e.g. 'google'
    method:str='none', # rerank method: none | lexical | flashrank | auto
    pages:int=1, # result pages to pull from each backend (raises the candidate pool)
    region:str='auto', # ddgs region, or 'auto' to read the country off the query
    timelimit:str=None, # d | w | m | y — only results from the last day/week/month/year
    timeout:int=8, **kw
)->list: # extra kwargs passed to ddgs (max_results, safesearch, page)

Search the web, merging text results from several engines with RRF and optionally reranking them.

`google`

google loads Google’s results page in the stealth browser and reads the results off it. The ranking is Google’s own. It is slower than search and needs scrapling’s browsers. It takes the same region and timelimit as search. The country biases the ranking and does not exclude other countries. The time limit uses Google’s own filter. Its results have the same fields as search’s, with the snippet under content. When Google returns no results, google returns an empty list. When the browser fails, it returns an empty list and prints a warning.

google

def google(
    q:str, # search query
    n:int=10, # number of results
    lang:str=None, # `hl`; defaults to the language half of `region`
    region:str='auto', # ddgs-style region, or 'auto' to read the country off the query
    timelimit:str=None, # d | w | m | y — Google's own `tbs=qdr:` filter
)->list:

Search Google itself through the stealth browser and return results in the same shape as search.

`research`

research turns a question into a set of readable sources in these steps:

plan, from fossick.quality, splits a question that asks several things into one search per part, and keeps the original question as the first search.
Each search runs with search, or with google when engine='google'. It interleaves their results, taking one from each in turn. Each part of the question gets sources.
With curated=True, the default, curate removes sites that copy other sites’ content and moves authoritative sources up. research drops copycats before it downloads any page.
It downloads the candidates in order with fetch(auto=True). Each batch has as many pages as n still lacks. It stops when it has n pages with at least 400 characters of readable markdown. A page that failed, was blocked, or was too short goes into dropped with its reason. The next candidate takes its place.
With focused=True, focus cuts each page down to about chars characters of the passages that best answer the question.

It returns a dictionary with the query, the sources, the digest, the dropped pages, the region it used, the curation report and the plan. Each source holds its title, href, the fetch tier that worked, its publication date, the queries it answered and its markdown in md. The digest joins the sources, separated by ---, each headed by its title, URL, publication date when known, and the part of the question it answers.

Three helpers work on their own. page_date finds the date a page says it was published and returns it as YYYY-MM-DD, or None. It reads the JSON-LD data, the <meta> tags (the names in DATE_META), a <time> element, or a /2018/09/ style URL. usable checks that a fetched page is a 200 response, is not a bot wall, and has at least min_chars characters of readable markdown. min_chars=0 skips the length check. focus keeps the parts of a long page that answer a query. It keeps the opening blocks up to 30% of the budget, because reference pages often answer in their first paragraphs. It then adds the blocks that score best for the query. It returns them in their original order with … marking the gaps.

The cell below this one tests paging with stand-in engines.

# pagination: ranks run on across pages, and page 2's repeats of page 1 don't get counted twice
_seen = []
def _fake(be, q, region='us-en', page=1, timeout=8, tries=2, **kw):
    _seen.append((be, page))
    start = (page-1)*2
    return [dict(title=f'{be}{i}', href=f'https://e.com/{be}/{i}', body='', engine=be, rank=i-start+1)
            for i in range(start, start+3)]        # 3 hits/page, overlapping the next page by one
_real, _engine_hits = _engine_hits, _fake
try:
    one = _backend_lists('q', ('brave',), pages=1)
    two = _backend_lists('q', ('brave',), pages=2)
    later = _backend_lists('q', ('brave',), page=3)
finally: _engine_hits = _real

assert sorted(_seen[:3]) == [('brave',1), ('brave',1), ('brave',2)], _seen   # one job per (backend, page)
assert _seen[3:] == [('brave',3)]                                        # `page` is where paging starts
assert len(one[0]) == 3 and len(two[0]) == 5                             # 3+3 minus the one repeat
assert [h['rank'] for h in two[0]] == [1,2,3,4,5]                        # continuous, not 1,2,3,1,2,3
assert len({h['href'] for h in two[0]}) == 5                             # deduped within the backend

research

def research(
    q:str, # search query
    n:int=5, # number of readable sources to gather
    engine:str='search', # 'search' (fused metasearch) or 'google' (stealth browser ranking)
    sel:str=None, # CSS selector to narrow each page before markdown
    chars:int=4000, # max markdown chars kept per page
    auto:bool=True, # auto-escalate fetching (plain->heavy->stealthy->session) per page
    region:str='auto', # ddgs region, or 'auto' to read the country off the query
    timelimit:str=None, # d | w | m | y — only sources from the last day/week/month/year
    curated:bool=True, # drop mirror domains and rank by source authority before reading
    intent:str='auto', # authority class ('policy', 'docs', …), 'auto' to read it off `q`
    facets:int=4, # searches to fan out over when `q` holds more than one question; 1 disables
    rewrite:NoneType=None, # optional `f(q) -> list[str]` (a model) in place of the heuristic planner
    focused:bool=True, # keep query-relevant passages instead of the first `chars` chars
    candidates:int=None, # how many hits to draw on when backfilling (default 3n)
    pages:int=1, # result pages per backend (raise it when `candidates` exceeds one page)
    concurrency:int=8, **kw
)->dict: # extra kwargs passed to fetch()

Search for q, read the top n readable results, and return them with a markdown digest that cites each one.

focus

def focus(
    md:str, # page markdown
    q:str, # query to keep relevant to
    chars:int=4000, # character budget
    reserve:float=0.3, # fraction of the budget spent on the top of the page before ranking
    lead:float=0.6, # extra score for blocks near the top (intros define things)
)->str:

Keep the parts of md that best answer q, in their original order, within chars characters.

usable

def usable(
    pg, min_chars:int=400
)->bool:

Return True when a fetched page is a 200 response, not a bot wall, and has at least min_chars of readable markdown.

page_date

def page_date(
    page
)->str:

Return the publication date a page states, as YYYY-MM-DD, or None.

#: `focus` counts the separators and omission markers it adds, so the result never exceeds `chars`
_page = '\n\n'.join('b%02d' % i + 'a' * 96 for i in range(40))
assert all(len(focus(_page, 'b07', chars=c)) <= c for c in (4000, 1000, 300))

Tests

The first cells run offline: they test empty queries, the Google results parser, URL normalisation and RRF, BM25 and focus, bot wall detection, region inference, that region and timelimit reach every engine, page_date, and the shape of research’s result. The cells after them run real searches and are skipped by the tests; their saved output shows what each call returns.

# empty query is a no-op for every entry point (no network)
assert search('') == []
assert research('')['sources'] == []
assert rrf([]) == [] and rerank('x', []) == []

# Google SERP parser works on a static fixture (no browser/network)
_serp = ('<html><body><div class="MjjYud"><div class="g">'
         '<a href="https://ex.com/p"><h3>Example Page</h3></a>'
         '<div class="VwiC3b">A short snippet.</div></div></div>'
         '<div><a href="https://www.google.com/aclk"><h3>Ad</h3></a></div></body></html>')
_g = _parse_google(_serp, 5)
assert len(_g) == 1, f'expected 1 organic result, got {len(_g)}'   # google.com ad link filtered out
assert _g[0]['href'] == 'https://ex.com/p' and _g[0]['title'] == 'Example Page'
assert _g[0]['content'] == 'A short snippet.'

# urls that differ only by scheme/www/trailing slash/tracking params are one page
assert norm_url('https://www.x.com/a/?utm_source=t#frag') == norm_url('http://x.com/a')
assert norm_url('https://x.com/a') != norm_url('https://x.com/b')

_g = [dict(href='https://x.com/1', rank=1, engine='google'), dict(href='https://x.com/8', rank=8, engine='google')]
_b = [dict(href='https://x.com/8', rank=8, engine='brave'),  dict(href='https://x.com/2', rank=2, engine='brave')]

# two urls, one backend each: the better-ranked one wins. ddgs ties them and orders by thread completion.
_solo = rrf([_g[:1], _b[1:]])
assert [h['href'] for h in _solo] == ['https://x.com/1', 'https://x.com/2'], _solo

# agreement across backends still outranks a single strong hit — that part of ddgs' instinct is right
_f = rrf([_g, _b])
assert _f[0]['href'] == 'https://x.com/8' and _f[0]['engines'] == ['google', 'brave']
assert len(_f) == 3                                                    # deduped across backends

# nothing is hoisted by domain: wikipedia earns its place from rank like everything else
_w = rrf([[dict(href='https://en.wikipedia.org/wiki/Rust', rank=7, engine='google'),
           dict(href='https://actix.rs/', rank=1, engine='google')]])
assert _w[0]['href'] == 'https://actix.rs/', _w

# BM25 ranks the doc that actually covers the query terms
_docs = ['sqlite wal mode benchmark write ahead log', 'a recipe for banana bread', 'wal mode']
_s = bm25('sqlite wal mode benchmark', _docs)
assert _s[0] == max(_s) and _s[1] == 0.0, _s

# rerank pulls the on-topic hit up past a higher-retrieval-score off-topic one
_hits = [dict(href='https://a.com/cake', title='Banana bread', body='baking', score=1.0),
         dict(href='https://b.com/sqlite-wal-benchmark', title='SQLite WAL benchmark', body='wal vs journal', score=0.9)]
assert rerank('sqlite wal benchmark', _hits, method='lexical')[0]['href'].startswith('https://b.com')

# focus() keeps the passage that answers the query, not the first N chars
_md = ('Cookie notice. We use cookies.\n\n' + 'Navigation menu home about contact.\n\n'*8 +
       'WAL mode allows readers and writers to proceed concurrently, unlike the rollback journal.\n\n' +
       'Subscribe to our newsletter.\n\n'*8)
_f = focus(_md, 'wal mode concurrent readers writers', chars=200)
assert 'concurrently' in _f and len(_f) <= 260, _f
assert 'concurrently' not in _md[:200]      # head truncation would have missed it

# a Cloudflare interstitial is a 200 that must NOT reach the corpus
class _P:
    def __init__(self, status=200, html=''): self.status, self.html_content, self.html = status, html, html
assert not usable(_P(200, '<html><head><title>Just a moment...</title></head><body>Enable JavaScript and cookies to continue</body></html>'))
assert not usable(_P(403, '<html>' + 'x'*500 + '</html>'))
assert not usable(None)
assert usable(_P(200, '<html><body>' + 'real prose about sqlite. '*40 + '</body></html>'))
assert _why(_P(429)) == 'http 429' and _why(None) == 'fetch failed'

# `usable` enforces `min_chars`.
_pg = _P(200, '<html><body>' + 'short prose about sqlite. '*40 + '</body></html>')
assert usable(_pg, min_chars=100) and usable(_pg, min_chars=0)
assert not usable(_pg, min_chars=5000)

# where and what-language are separate questions with separate answers
assert infer_country('rebuild my garage in australia') == 'au'
assert infer_country('garage rebuild sydney quotes') == 'au'           # cities count
assert infer_country('planning permission in the UK') == 'uk'
assert infer_country('compare australia and germany building codes') is None   # a comparison: neither
assert infer_country('garage rebuild cost') is None
assert infer_country('give us a quote') is None                        # bare 'us' is the pronoun
assert infer_country('australianism') is None                          # word boundaries

assert infer_language('what is the best way to do this for my house') == 'en'
assert infer_language('welche Vorschriften gelten für eine Garage in Österreich') == 'de'
assert infer_language('garage') is None                                # one word is not evidence
assert infer_language('la') is None                                    # shared across languages

# the join: country from the subject, language from the asker
assert infer_region('I want to rebuild my garage in australia, what is the regulation') == 'au-en'
assert infer_region('garage rebuild sydney') == 'au-en'
assert infer_region('do I need planning permission for a garage in the UK') == 'uk-en'
assert infer_region('what is the cost of living in france') == 'fr-en'  # French sources, English pages
assert infer_region('welche Vorschriften gelten für eine Garage in Österreich') == 'at-de'
assert infer_region('aktuelle KI Regulierung Österreich Unternehmen 2026') == 'at-de'  # no function words: fall to the country
assert infer_region('compare australia and germany building codes') == 'us-en'         # abstains to the default
assert infer_region('garage rebuild cost') == 'us-en'
assert infer_region('', dflt='au-en') == 'au-en' and infer_region(None) == 'us-en'
assert _region('garage in australia', 'uk-en') == 'uk-en'              # explicit beats inferred

# `region` and `timelimit` reach every backend.
_calls = []
def _spy(be, q, region='us-en', page=1, timeout=8, tries=2, **kw):
    _calls.append(dict(be=be, region=region, **kw))
    return [dict(title=be, href=f'https://e.com/{be}', body='', engine=be, rank=1)]
_real, _engine_hits = _engine_hits, _spy
try:
    search('sqlite wal mode', timelimit='y')
    assert [c['timelimit'] for c in _calls] == ['y']*len(TEXT_BACKENDS), _calls
    assert {c['region'] for c in _calls} == {'us-en'}

    _calls.clear(); search('rebuild my garage in australia')
    assert {c['region'] for c in _calls} == {'au-en'}, _calls
    assert not any('timelimit' in c for c in _calls)          # absent, not None: ddgs defaults it

    _calls.clear(); search('garage in australia', region='uk-en', timelimit='m', safesearch='off')
    assert {c['region'] for c in _calls} == {'uk-en'}
    assert all(c['timelimit'] == 'm' and c['safesearch'] == 'off' for c in _calls)

    _calls.clear(); assert search('  ') == [] and _calls == []
finally: _engine_hits = _real

# `page_date` reads JSON-LD, `<meta>` tags, `<time>` elements and URLs, and returns None for an undated page.
class _D:
    def __init__(self, html='', url=''): self.html, self.url = html, url

_ld = lambda s: _D(f'<script type="application/ld+json">{s}</script>')
assert page_date(_ld('{"@type":"Article","datePublished":"2024-03-07T09:00:00Z"}')) == '2024-03-07'
assert page_date(_ld('{"@graph":[{"@type":"WebSite"},{"datePublished":"2019-11-02"}]}')) == '2019-11-02'
assert page_date(_ld('{"datePublished":"not a date","dateModified":"2024-05-05"}')) == '2024-05-05'
assert page_date(_D('<meta property="article:published_time" content="2022-06-15T10:11:12+00:00">')) == '2022-06-15'
assert page_date(_D('<meta name="DC.date.issued" content="2021-1-5">')) == '2021-01-05'
assert page_date(_D('<meta name="viewport" content="width=1"><meta name="date" content="2025-12-31">')) == '2025-12-31'
assert page_date(_D('<time datetime="2023-08-01">last August</time>')) == '2023-08-01'
assert page_date(_D('', 'https://blog.example.com/2018/09/wal-mode')) == '2018-09-01'   # month only -> day 01
assert page_date(_D('<p>nothing dated here</p>', 'https://e.com/x')) is None

# a malformed JSON-LD block must not swallow the meta tag underneath it
assert page_date(_D('<script type="application/ld+json">{oops</script>'
                    '<meta name="pubdate" content="2020-02-29">')) == '2020-02-29'

# the month/day alternation has to prefer the longest branch, or 2019-11-02 reads as 2019-01-01
assert _iso('2019-11-02') == '2019-11-02' and _iso('2025-12-31') == '2025-12-31'
assert _iso('v1.2 build 7') is None

# research() reports the region it resolved, and keeps one shape whether or not it found anything
_r = research('')
assert _r['sources'] == [] and _r['digest'] == ''
assert sorted(_r) == ['curation', 'digest', 'dropped', 'plan', 'query', 'region', 'sources']
assert _r['plan'] == []
assert _r['region'] == 'us-en' and research('', region='au-en')['region'] == 'au-en'

# text search — every backend queried in parallel, fused with RRF, then reranked
for r in search('fasthtml python web framework', n=5):
    print(f"{r['score']:.4f} {str(r.get('engines')):40s} {r['title'][:50]} — {r['href']}")

0.0320 ['brave', 'startpage']                   Python FastHTML: A Beginner’s Guide with Examples  — https://www.datacamp.com/tutorial/python-fasthtml
0.0318 ['brave', 'startpage']                   FastHTML - Modern web applications in pure Python — https://fastht.ml/
0.0315 ['brave', 'startpage']                   FastHTML: Revolutionizing Web Development with Pyt — https://www.geeksforgeeks.org/python/fasthtml-modern-web-application-in-pure-python/
0.0315 ['brave', 'startpage']                   python-fasthtml · PyPI — https://pypi.org/project/python-fasthtml/
0.0296 ['brave', 'startpage']                   GitHub - AnswerDotAI/fasthtml: The fastest way to  — https://github.com/AnswerDotAI/fasthtml

for r in search('red panda', 'images',n=3): print(r['title'], '—', r['image'])

Happy Red Panda — https://animalfactguide.com/wp-content/uploads/2020/12/red_panda_220-scaled.jpg
Cute Red Panda — https://i.natgeofe.com/k/63b1a8a7-0081-493e-8b53-81d01261ab5d/red-panda-full-body_16x9.jpg
Red Panda Animal Endangered Red Pandas Seized From Smuggler In Laos, — https://cincinnatizoo.org/wp-content/uploads/2012/04/Red-Panda-Image-for-Endangered-Excursion-scaled.jpg

# news search
for r in search('open source LLMs', 'news', n=3, pages=2): print(r['date'], r['title'])

2025-03-20T10:45:00+00:00 The Open-Source LLM Revolution: Transforming Enterprise AI For A New Era
2026-07-25T00:15:00+00:00 Nvidia, other tech giants caution against open-source AI ban in open letter
2023-06-20T00:00:00+00:00 Open-source AI chatbots are booming — what does this mean for researchers?

for r in videos(q='aditya hrudayam', n=1): print(r)

ERROR: query "aditya hrudayam" page 1: Unable to download API page: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1018) (caused by CertificateVerifyError('[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1018)')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

search_yt error: ERROR: query "aditya hrudayam" page 1: Unable to download API page: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1018) (caused by CertificateVerifyError('[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1018)')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

# real Google ranking via stealth browser (slower; use when you need Google specifically)
for r in google('vedicreader website', n=5): print(r['title'], '—', r['href'])

[2026-08-09 15:39:05] INFO: Fetched (200) <GET https://www.google.com/search?q=vedicreader+website&hl=en&num=7&sei=dRJ4arGzHuGihvcP0anygAk> (referer: https://www.google.com/)

VedicReader — https://vedicreader.com/
VedicReader — https://vedicreader.com/a/lgn
vedicreader.com to help absorb vedic texts easily — https://www.reddit.com/r/AdvaitaVedanta/comments/1lpr29l/vedicreadercom_to_help_absorb_vedic_texts_easily/
VedSearch — Read the Vedas in Sanskrit, Hindi & English — https://vedsearch.org/
Ved Portal - Search & Read — https://www.xn--j2b3a4c.com/en/

# research(): search -> read the top n results -> one cited markdown corpus
res = research('what is the raft consensus algorithm', n=3)
assert set(res) == {'query', 'sources', 'digest', 'dropped'}, res.keys()
assert res['query'] and isinstance(res['sources'], list) and res['sources']
assert all({'title','href','md','tier'} <= set(s) for s in res['sources'])
assert res['digest']
print(len(res['sources']), 'sources |', len(res['digest']), 'chars |', len(res['dropped']), 'skipped')
for d in res['dropped']: print('  skipped:', d['reason'], d['href'])

[2026-08-09 15:39:10] INFO: Fetched (200) <GET https://www.geeksforgeeks.org/system-design/raft-consensus-algorithm/> (referer: https://www.google.com/)
[2026-08-09 15:39:10] INFO: Fetched (200) <GET https://raft.github.io/> (referer: https://www.google.com/)
[2026-08-09 15:39:10] INFO: Fetched (200) <GET https://en.wikipedia.org/wiki/Raft_(algorithm)> (referer: https://www.google.com/)

3 sources | 10171 chars | 0 skipped