WebRouter
An open source standard that connects your agent to the internet.
Connect your agent to every web tool through one interface: search, extract, live data, answer, research and browser sessions, across many providers. A router can pick the tool and provider for each task, trading off cost, latency and coverage for your domain and the providers you have keys for.
pip install "webrouter[all]"
from web_router import Provider, WebRouter
WebRouter().search("what is retrieval augmented generation") # no key needed
WebRouter(Provider.NIMBLE).search("...") # or name one
WebRouter(Provider.TAVILY).search("...") # same call, same result shape
WebRouter().extract("https://example.com") # page contents, also keyless
WebRouter().live_data("walmart_pdp", {"product_id": "..."}, provider="nimble") # a site's fields, fetched now
WebRouter(Provider.LINKUP).answer("...") # a written answer with its sources, in one call
WebRouter().call_vendor("datadog.logs.search", {"filter": {"query": "status:error"}}) # your own vendor accounts
WebRouter(Provider.NIMBLE).research("...") # an agentic run: minutes, not milliseconds
with WebRouter("local").open_browser() as browser: # a browser to drive, also keyless
browser.goto("https://example.com")
WebRouter(Provider.BROWSER_USE).browse("...") # a hosted browser agent does the task
Useful? Star the repo so more agent developers can find it. ⭐
Beta: the API may change before 1.0.
Every provider has its own SDK, its own parameter names and its own response
shape. "How many results" is max_results on Tavily, count on Brave and
You.com, numResults on Exa. The page text is content on Tavily, text on
Exa, snippet on Perplexity, and nested under contents on You.com. Swapping
backends means rewriting the call, the options and the parsing.
WebRouter normalizes all three:
response = router.search(query, max_results=5, search_depth="fast")
[result.content for result in response.results] # same field, every backend
The extract backends disagree just as widely. A page body is data.markdown on
Nimble, raw_content on Tavily, text on Exa, results[].content on Oxylabs
and body inside an envelope on Bright Data — and most of them take one URL per
call while two take a list:
pages = router.extract(response.urls[:5], format="markdown")
[document.content for document in pages.documents] # same field, every backend
Nothing is hidden to get there. Unmapped provider fields stay in
SearchResult.extra / ExtractedDocument.extra, the untouched payloads stay in
raw, and an option a backend cannot honour **raises instead of being quietly
dropped** — so a call never runs under weaker constraints than you asked for.
Status: early preview.search,extract,live_data(with
live_data_sourcesandlive_data_sourcebeside it),research,open_browser
andbrowseare implemented;benchmark()andbenchmark_extract()
compare search and extract backends on your own queries and URLs, and
evaluate() scores search accuracy with an LLM judge —
see Benchmarking providers and
Evaluating accuracy. The API may change before 1.0.
Install
Install the package from PyPI, with the extras you need:
pip install "webrouter[all]" # any virtualenv
uv add "webrouter[all]" # a uv project
pip install "webrouter[mcp,browser]" # the MCP server with its browser tools
Then put the keys you hold in your environment or a .env file; none is
required. .env.example names each variable.
From source, for contributors or to track main: the one prerequisite is
uv, which fetches
the right Python itself.
git clone https://github.com/Nimbleway/webrouter.git
cd webrouter
make setup # every extra: the library, the MCP server, the browser tools and a local Chromium
cp .env.example .env # then fill in only the keys you hold; none is required
Using a coding agent? Tell it *"clone https://github.com/Nimbleway/webrouter
and set it up"*. CLAUDE.md walks it through the rest. To use the
clone from another project, install it in editable mode
(uv add --editable "/path/to/webrouter[all]" or
pip install -e "/path/to/webrouter[all]"), so git pull && make setup in the
clone is the whole update.
You can install fewer extras by naming them in place of all:
search, extract, research, browser, mcp, evals and langchain.
Extras are named after the methods you call, not the backends behind them:
search brings in every provider that can serve one, so choosing between them
stays a runtime decision — WebRouter(Provider.BRAVE) — instead of something you have
to get right at install time.
Three extras are not methods. mcp names a transport — it installs the
MCP server, which exposes the methods above rather than
adding one of its own. It does not include browser: the server's browser_*
tools need Playwright, so they appear only with webrouter[mcp,browser].
evals installs the OpenAI and Anthropic SDKs behind the built-in judges that
evaluate() grades answers with; bring your own judge
and you do not need it. langchain installs langchain-core for the
LangChain tools — the agent loop and the model stay your own
install. browser is named after what it lets a browser
handle do — drive the page — rather than after open_browser, which needs no
Playwright: a vendor session opens over httpx, which [all] installs, and
local needs only the standard library and a Chrome or Chromium. remote
installs httpx for the built-in remote router,
for an install that carries no method extra; [all] includes it.
Quickstart
No key needed to try it — the default provider uses the backends that offer an anonymous tier:
from web_router import WebRouter
response = WebRouter().search("what is retrieval augmented generation", max_results=5)
for result in response.results:
print(result.title, result.url)
print(response.provider) # which backend actually answered
Name a provider once you have a key for one. Provider is an enum of the
built-in backends, and it is the form worth writing: your editor completes it,
and a typo is caught where you type it rather than at the first call.
from web_router import Provider, WebRouter
router = WebRouter(Provider.NIMBLE) # reads NIMBLE_API_KEY from the environment
response = router.search("who founded nimbleway", max_results=5, country="US")
router.search("...", provider=Provider.BRAVE) # per-call override
country is a good illustration of the contract: Nimble maps it, and a backend
that does not raises UnsupportedParameterError rather than quietly running the
search unfiltered.
Plain strings work anywhere a Provider does — WebRouter("nimble") is the
same call. The enum covers the built-ins, so the string form is what names a
backend you added with register_provider, or one that arrives from a config
file, an environment variable or a CLI flag. Prefer the enum when you are
writing the name yourself; reach for a string when something else supplies it.
Searching and then reading the pages you found is the two methods together:
response = router.search("latest nvidia earnings", max_results=5)
pages = router.extract(response.urls[:3], format="markdown")
for document in pages.documents:
print(document.url, len(document.content))
Async works the same way:
import asyncio
from web_router import AsyncWebRouter, Provider
async def main():
async with AsyncWebRouter(Provider.NIMBLE) as router:
response = await router.search("latest nvidia earnings", max_results=3)
pages = await router.extract(response.urls, format="markdown")
print(pages.urls)
asyncio.run(main())
Runnable examples
examples/ has a script per topic — routing, every error
and its fix, depth measured across backends, adding your own provider — and
most need no credentials, routing through the keyless backends —
browser_session.py drives a local Chromium
with none. main.py is a guided tour of search and extract
in one file:
uv run --env-file .env python examples/main.py
Provider support
Twelve search backends, seven extract backends, a template catalogue, five answer backends, three agentic research backends (plus Exa's, retired upstream), five browser-session backends and three browser agents. One call each. Switching between them is one argument.
A provider name serves the methods it serves and no others: brave searches
and cannot extract, oxylabs extracts and cannot search, nimble searches,
extracts, runs live data and researches, and browserbase opens
sessions, runs browse tasks and extracts.
Ask a method for another's backend and it says so, naming the backends that
method does serve.
Search
[search] installs all of these but AgentCore, which needs [aws]. The first column is what you pass to
WebRouter(...) to pick one at runtime. The string beside it is the same value,
and what response.provider and configured_providers() report back.
| Pass | Name | Provider | Backed by | Options mapped |
| --- | --- | --- | --- | --- |
| Provider.NIMBLE | nimble | Nimble | nimble-python | all but effort |
| Provider.TAVILY | tavily | Tavily | tavily-python | all but effort |
| Provider.YOU | you | You.com | direct HTTP | all but effort |
| Provider.BRAVE | brave | Brave | direct HTTP | all but effort; search_depth at lite only |
| Provider.EXA | exa | Exa | direct HTTP | all but locale |
| Provider.PARALLEL | parallel | Parallel | direct HTTP (/v1/search, mode=fast unless effort sets it) | all but locale; search_depth at lite and fast |
| Provider.PERPLEXITY | perplexity | Perplexity | direct HTTP | all of them; search_depth at lite and fast |
| Provider.LINKUP | linkup | Linkup | direct HTTP | max_results, include_domains, exclude_domains, recency, effort |
| Provider.WEBIQ | webiq | WebIQ (Microsoft AI Web Search) | direct HTTP | all but recency and effort |
| Provider.KEENABLE | keenable | Keenable | direct HTTP; keyless with allow_keyless=True (rate-limited per IP) | max_results, search_depth (lite, fast), include_domains (one), recency, effort |
| Provider.DEEPSEEK | deepseek | DeepSeek Search | direct HTTP | max_results (at most 50), search_depth (lite, deep), include_domains, exclude_domains, locale, recency, the last four sent inside the query, which must fit 128 characters |
| Provider.AGENTCORE | agentcore | AWS AgentCore Web Search, through your own AgentCore Gateway (setup) | MCP over HTTP, SigV4 via botocore ([aws]) | max_results (at most 25), include_domains, exclude_domains, recency (connector 1.2.0 or later). An end date is AgentCore's own provider_options={"filters": {"publishedDateFilter": {"to": ...}}}. Queries up to 200 characters. AWS's terms: show each result's citation, and do not store results in bulk |
| Provider.YOU_MCP | you-mcp | You.com MCP | direct HTTP, no key | all but effort |
| Provider.EXA_MCP | exa-mcp | Exa MCP, its web_search_advanced_exa tool | direct HTTP, no key | all but locale |
| Provider.PARALLEL_MCP | parallel-mcp | Parallel Search MCP | direct HTTP, no key | max_results (applied to the response: the tool ignores a count) |
| Provider.AUTO (default) | auto | — | every backend you hold a key for, in the order the frozen benchmark results rank them, then the keyless ones — so a key held only for research serves searches too, and those calls bill; see Working with no credentials | every option some member maps, each request routed to a member that maps what it sets |
A backend maps an option only where its API has a documented field or query
operator for it; Search options says how each one does, and
contracts/search/ holds the exact fields sent. Mapping another is a
small PR and the most useful thing to contribute — see
CONTRIBUTING.md.
Extract
[extract] installs all of these, and configured_extract_providers() reports
which ones this environment can use.
| Pass | Name | Provider | Credentials | Formats |
| --- | --- | --- | --- | --- |
| Provider.NIMBLE | nimble | Nimble Extract | NIMBLE_API_KEY | markdown, html |
| Provider.TAVILY | tavily | Tavily Extract | TAVILY_API_KEY (keyless only through auto) | markdown, text |
| Provider.EXA | exa | Exa Contents | EXA_API_KEY | text |
| Provider.LINKUP | linkup | Linkup Fetch | LINKUP_API_KEY | markdown, html |
| Provider.FIRECRAWL | firecrawl | Firecrawl Scrape | FIRECRAWL_API_KEY, or keyless with allow_keyless=True (capped per IP per day) | markdown, html |
| Provider.OXYLABS | oxylabs | Oxylabs Web Scraper | OXYLABS_USERNAME + OXYLABS_PASSWORD | markdown, html |
| Provider.BRIGHTDATA | brightdata | Bright Data Web Unlocker | BRIGHTDATA_API_TOKEN + BRIGHTDATA_ZONE | markdown, html |
| Provider.WEBIQ | webiq | WebIQ Browse (Microsoft AI) — pages in WebIQ's own index | WEBIQ_API_KEY | markdown, text |
| Provider.KEENABLE | keenable | Keenable Fetch | KEENABLE_API_KEY, or keyless with allow_keyless=True (rate-limited per IP) | markdown |
| Provider.DEEPSEEK | deepseek | DeepSeek Fetch | DEEPSEEK_API_KEY | markdown |
| Provider.JINA | jina | Jina Reader | none (JINA_API_KEY raises the rate limit) | markdown, html, text |
| Provider.BROWSERBASE | browserbase | Browserbase Fetch — a plain fetch, never a rendered page; the one backend that maps profile | BROWSERBASE_API_KEY | markdown, html |
| Provider.AUTO (default) | auto | — | the original backends you hold a key for, then the keyless ones, then Linkup, Exa, WebIQ, Keenable, DeepSeek and Browserbase | markdown (default), html, text, each routed to a member that returns it; profile routed to one that maps it |
| Provider.MAX | max | — | the same members as auto, all at once; the first document back wins, and every fetch started is paid for -- see The fastest answer any backend can give | as auto |
Two need more than a key. Oxylabs authenticates with the username and password created under Web Scraper API in its dashboard — not the ones you sign in with — and Bright Data needs the zone that routes the request beside its token. Neither has a default worth guessing, so a missing one is an error naming the variable to set.
Nimble, Tavily, Jina, Linkup, Firecrawl, Oxylabs, Bright Data and Browserbase
were verified live — Oxylabs and Bright Data on 2026-11-03, Browserbase (both
formats, and profile through the whole flow) by hand on 2026-10-29 and by
the automated live tests on 2026-11-08. **Exa's extract
mapping is taken from documentation** and has not been confirmed against a
real response; its integration tests skip themselves until a working key is
available.
Reading a page behind a login
profile is the extract side of the browser's profile: a saved identity on
the backend — its cookies and storage — whose cookies are sent with each fetch.
Log in once in a browser session that persists to it, release the session, and
the fetches that name the same id read the page as that user:
with router.open_browser(provider="browserbase", profile=context_id, persist_profile=True) as browser:
browser.goto("https://example.com/login")
... # type the credentials, submit
response = router.extract("https://example.com/account/orders", provider="browserbase", profile=context_id)
The fetch never writes cookies back, so an expired login means another
session. Browserbase is the one backend that maps it, through its Contexts; a
backend without saved profiles raises UnsupportedParameterError rather than
fetching anonymously, and auto and max pass such members over. The MCP
server never offers it: a logged-in identity is set in code, not from a
conversation.
Live data
A live data source is a vendor's ready-made scraper for one site -- Nimble's
Extract Templates, Apify's Actors, Bright Data's scrapers, the Oxylabs Web
API's dedicated scrapers. Given its own inputs it fetches the site for this call
and answers with structured records. That is the rule a backend joins under:
it fetches on request. Data a vendor collected earlier and serves from storage
is that vendor's own API, reached through call_vendor(). There is no keyless
tier and no auto: a run fetches and parses on the backend's account, and a
source's name belongs to one vendor, so provider is required on all three
calls.
| Pass | Name | Provider | Credentials | Page formats beside the records |
| --- | --- | --- | --- | --- |
| Provider.NIMBLE | nimble | Nimble Extract Templates | NIMBLE_API_KEY | markdown, html |
| Provider.APIFY | apify | Apify Store Actors | APIFY_TOKEN | — |
| Provider.BRIGHTDATA | brightdata | Bright Data scrapers | BRIGHTDATA_API_TOKEN | — |
| Provider.OXYLABS | oxylabs | Oxylabs Web API dedicated scrapers | OXYLABS_WEB_API_KEY (a Web API key, not the Realtime username and password extract uses; in provider_config["oxylabs"] it is web_api_key, as api_key there is extract's password) | — |
Nimble was verified live on 2026-10-22: a listing, a description and a run on
a public test page. Apify and Bright Data were verified live on 2026-11-10:
each one's catalogue, a search, a description and a run (Apify's Google Maps
Actor, Bright Data's amazon_products). **The Oxylabs Web API is written from
its documentation** and has not yet been called live. Their integration tests
skip themselves until keys are available. Listing and describing bill nothing
on any of them; what a run bills is each vendor's own rate, recorded dated and
sourced in benchmarks/providers/PRICING.md
from the price list web_router.prices reads (see
Estimated prices). See Live data.
Answer
[answer] installs these. One question, answered in one call: the backend
searches and writes the answer itself. No keyless tier, since every backend
bills the call; auto tries the ones you hold a key for, in the order make chains ranks them on the
answer benchmark. See
Asking for an answer for what an unnamed call does.
| Pass | Name | Provider | Credentials | Options mapped | Citations tied to claims |
| --- | --- | --- | --- | --- | --- |
| Provider.PERPLEXITY | perplexity | Perplexity Agent API, fast effort | PERPLEXITY_API_KEY | include_domains or exclude_domains (not both), country, recency | yes: [N] markers, in citation.field |
| Provider.YOU | you | You.com Answer API | YOU_API_KEY | include_domains or exclude_domains (not both), country, recency (week or wider: for day it cited pages days old) | yes: [[1, 2]] markers, in citation.field |
| Provider.PARALLEL | parallel | Parallel Chat API (beta), lite | PARALLEL_API_KEY | none: the Chat API takes no source filter | no: its basis backs the answer as a whole |
| Provider.EXA | exa | Exa Answer API | EXA_API_KEY | none: no source filter, and userLocation showed no clear effect | yes: [N] markers, in citation.field |
| Provider.LINKUP | linkup | Linkup Search, sourcedAnswer | LINKUP_API_KEY | include_domains, exclude_domains, recency | yes: [N] markers, in citation.field |
| Provider.BRAVE | brave | Brave Answers API | BRAVE_ANSWER_API_KEY (the Answers plan; not the search key) | none: no source filter, and language showed no effect | yes: each citation's span in the answer, in citation.extra |
All verified live on 2026-11-01, every mapped option seen to apply. The
medians and stated costs below are from the answer scan of 300 SimpleQA
questions on 2026-11-01, graded by openai:gpt-4o.
perplexityanswers in about two seconds (median 1.9 s over 300 questions) and cites inline as[2]; a
field, and the
response states its own cost under raw[0]["usage"]["cost"] (about $0.0012
an answer over 300 questions). country is its user_location: an emergency-number question
answered 999 for GB and 911 for US.
youanswers in about two seconds (median 1.7 s over 300 questions). Itscitationsare only the sources the
[[1, 2]]; each carries in field the whole marker
group that first names it ([[1, 2]], where other backends give one [N]), and its title comes from the pages You.com considered. It takes one
domain list, not both: asking for both raises before the call. language was
accepted but showed no effect, so locale is not mapped.
parallelanswers in about six seconds (median 6.3 s over 300 questions) with thelitemodel, the fastest
basis ties its sources to the answer (speed returned none). Each
citation carries the basis's confidence in extra; its reasoning stays in
raw. Its Chat API is in beta and takes no source filter, so every option
is refused.
exaanswers in about 1.3 seconds (median over 300 questions) and cites inline as[1], numbering its
raw[0]["costDollars"] ($0.005 an
answer). Its Answer API has no source filter, so every option is refused.
linkupanswers in about a second (median 0.9 s over 300 questions) and lists every page it read; it is
[N], numbering them from 1, and a
page read but never cited has no field. recency is its fromDate, and Linkup filters on the date it holds
for a page, which is not always the publication date.
braveanswers in two to three seconds and was verified live on 2026-11-03, outside the scan. It returns its
and tags taken out; each tag is a citation, in order,
with its number and its start_index / end_index into the answer in extra, and Brave writes no marker
into the text, so field is unset. Its tag, in raw, states the call's cost: $0.054 for one
question, almost all of it input tokens, about ten times the others. Every option is refused.
Choosing one. On 300 SimpleQA questions graded by one judge, all five land
at 94-95% and their intervals overlap, so accuracy does not separate them --
the answer scan
reruns it. They differ in speed (Linkup, Exa, You.com and Perplexity in one to
two seconds, Parallel in about six), in the options they map (above), in
whether the answer ties each source to a sentence, and in cost: Perplexity and
Exa state theirs in the response; PRICING.md
lists the rest.
Research
[research] installs all of these. Unlike the other two methods there is no
keyless tier: every agentic backend bills for the run, so auto here tries the
backends you hold a key for and then stops, naming the variables that would fix
it.
| Pass | Name | Provider | Credentials | Options mapped | Structured output |
| --- | --- | --- | --- | --- | --- |
| Provider.NIMBLE | nimble | Nimble Web Search Agent | NIMBLE_API_KEY | all of them | yes, and enrichment |
| Provider.PARALLEL | parallel | Parallel Task API | PARALLEL_API_KEY | effort, output_schema | yes |
| Provider.PERPLEXITY | perplexity | Perplexity Agent API | PERPLEXITY_API_KEY | effort | not mapped yet |
| Provider.LINKUP | linkup | Linkup Research | LINKUP_API_KEY | effort, output_schema | yes |
| Provider.YOU | you | You.com Research API | YOU_API_KEY | effort | not mapped yet |
| Provider.AUTO (default) | auto | — | Nimble, Parallel, Perplexity, Linkup or You.com, whichever you hold, in that order | effort, output_schema | yes |
Nimble is the only backend WebRouter maps the whole surface for: source
guidance (include_domains / exclude_domains) and enrichment inputs
(records) are mapped for it alone so far. Parallel's Task API has a
source policy that is not
mapped yet. Ask another backend for either and it says so rather than running
without it.
Perplexity's and You.com's structured output is not mapped until a live run has
checked it, so neither declares output_schema — asking it for one raises
UnsupportedParameterError rather than handing back prose under the name of a
schema, which is the failure mode that quietly rots an agentic pipeline.
Exa's Research API has been retired upstream — every path answers 410 RESEARCH_RETIRED — so WebRouter no longer ships an Exa research provider. Exa search and extract are unaffected.
How far each of the others has been verified, since it differs and the difference matters:
| Provider | Verified how |
| --- | --- |
| parallel | Live. Real runs against the Task API, and every recorded payload in its tests is a real response. |
| nimble | Against the nimble-python SDK's generated types, produced from its OpenAPI spec — stronger than prose docs, weaker than a live call. Its max sends x-high, which Nimble's docs put at 15 to 30 minutes, longer than research()'s default 600-second wait: pass a larger timeout, or use submit_research() and poll, rather than leave a paid run going after the wait gives up. |
| perplexity | Live. On 2026-09-30 the chat endpoint's deep research model answered 403 ("Sonar is now the Agent API"), so research moved to /v1/responses: a background run was followed from queued through in_progress to completed, presets low, medium and xhigh were accepted, and a cancel left a run cancelled. Its payloads are the ones in the tests. |
| linkup | Live. Runs at effort="low" (reasoningDepth S, $0.25 each) were followed from start to answer, one of them with an output_schema, whose result was the schema's object alone; their payloads are the ones in the tests. Efforts above low are documentation-derived. Linkup's docs put max (XL) at 10 to 20 minutes and high (L) at 5 to 10, longer than research()'s default 600-second wait: pass a larger timeout, or use submit_research() and poll, rather than leave a paid run going after the wait gives up. |
| you | Live. On 2026-10-16 a background run at effort="low" (lite) was followed from queued through running to completed on api.you.com (the same path on the search host answers 403); an unknown research_effort was a 422 naming lite, standard, deep, exhaustive and frontier. Its payloads are the ones in the tests. You.com offers no cancel, so cancel() returns False. max maps to exhaustive: frontier, background-only and priced well above it, is reachable as provider_options={"research_effort": "frontier"}. Efforts above low are documentation-derived. |
Each has an integration test that skips itself until a credential is there.
Browser sessions and browse
open_browser() has five backends and auto, and browse() three and
auto. Their tables sit with the methods they serve, because each comes with
more to say than a row: see Session backends and
Browse backends.
Working with no credentials
auto is the default for every method, so a router you never configured still
searches, extracts and opens a browser — the Quickstart's first example is exactly that. It tries the backends
you have a key for first — a credential in the environment says which one you
want — and falls back to the anonymous tiers when there is none: Tavily keyless,
then You.com's, Exa's and Parallel's open MCP servers, then Keenable's keyless tier for a search; Tavily keyless,
Jina Reader, Firecrawl's keyless tier, then Keenable's for an extract; local for a browser
session. Research, browse, answer() and live data have no
anonymous tier, because every call bills; with no key they raise, naming the
variables that would fix it. This is the one list of keyless backends; the MCP
server and .env.example point here. response.provider names whichever one served it,
never "auto", so you can always tell. The anonymous tiers are rate-limited, which is
why it is a chain: when one is throttled, the next answers.
Every backend you hold a key for is a candidate, in a fixed order:
| Method | Keyed members, in order | Then keyless | Then members added since, with a key |
| --- | --- | --- | --- |
| search | Perplexity, Nimble, WebIQ, Linkup, Tavily, Exa, Parallel, Brave, You.com | Tavily keyless, You.com MCP, Exa MCP, Parallel MCP, Keenable keyless | Keenable, DeepSeek, AgentCore |
| extract | Nimble, Tavily, Firecrawl, Oxylabs, Bright Data | Tavily keyless, Jina Reader, Firecrawl keyless, Keenable keyless | Linkup, Exa, WebIQ, Keenable, DeepSeek, Browserbase |
| research | Nimble, Parallel, Perplexity, Linkup, You.com | nothing — every run bills | — |
Search's keyed order is generated, not hand-written: make chains ranks the
backends by accuracy on committed benchmark results; only backends too close in
accuracy to tell apart are ordered by cost, then speed.
benchmarks/providers/
explains the method, and how close the top backends are. Every
key you hold comes before the free tier — so a key you hold only for research,
Perplexity's say, serves your plain searches too. Keenable, DeepSeek and AgentCore have
no rows in those results, so they come after the keyless tier until a scan places them: with a gateway set up, a plain
WebRouter().search() is still served by a keyless member first, so name Provider.AGENTCORE to reach it. For extract, the backends that
joined later come after the keyless tier, so holding one more key never moves a
call that was already being served. Research has no free tier, so Linkup and then
You.com simply follow the others there. None of extract's, research's or browse's orders is
ranked: extract's is measured on pages returned, not graded, and research's and browse's are
unmeasured and hand-ordered.
To choose your own order, pass a chain.
auto routes each request to a member that can serve it. It accepts every
option some member maps, and before trying a member it reads that member's
class, which costs no credential and sends nothing. A member that does not map
an option you set, or refuses the value (a search_depth level it lacks, a
country off its list), is passed over for that call: ask for
search_depth="deep" with keys for only Perplexity and Brave, neither of which
returns whole pages, and the search goes to the keyless tier, where Exa MCP does.
The keyless tier between them maps every search option. When no member is left
it raises, naming what each one lacked, rather than running the query without
the option:
router = WebRouter(provider_config={"auto": {"chain": {"search": ["perplexity", "brave"]}}})
router.search("...", search_depth="deep")
UnsupportedParameterError: Provider 'auto' does not support search_depth='deep'. No backend
in its chain maps it, so nothing was searched: perplexity (will not serve
search_depth='deep'), brave (will not serve search_depth='deep'). ...
Among the members that can serve the call, a failure — a missing key, a refused request, a rate limit — moves on to the next, as it always has. So does an empty answer: a search with no results and no answer, or an extract whose every document is blank. When every member answers empty, you get the first empty answer, not an error. A call that ends empty has tried every member that can serve it, one after another, and each one you hold a key for bills for it. To stop at the first answer, empty or not:
router = WebRouter(provider_config={"auto": {"fallback_on_empty": False}})
That switch is the library's only; the MCP server always falls through.
For extract the same goes for the format, which every backend, auto
included, pins to markdown when you name none, rather than each applying
its vendor's own default. html passes over Tavily and Exa, and text goes to
Tavily, Exa or Jina Reader. A format is never relabelled: a member that cannot return it is
not asked.
Choosing the order yourself
chain in provider_config["auto"] replaces the default order. Keyed by
method, it sets that method's chain and leaves the others on their defaults:
router = WebRouter(
provider_config={
"auto": {
"chain": {
"search": ["perplexity", "nimble", "brave", "you-mcp"],
"extract": ["exa", ("tavily", {"allow_keyless": True}), "jina"],
}
}
}
)
Each member is a name, or (name, kwargs) for one that needs its own settings.
A member is built with its own provider_config entry, the chain's kwargs laid
over it, so a key passed in code reaches it through auto exactly as it does by
name. One backend first and another when it fails is a chain of two:
router = WebRouter(
provider_config={
"nimble": {"api_key": nimble_key},
"firecrawl": {"api_key": firecrawl_key},
"auto": {"chain": {"extract": ["nimble", "firecrawl"]}},
}
)
router.extract(url) # Nimble; Firecrawl when Nimble fails. response.provider says which.
The chain replaces the default whole, keyless tail included, and a member still
needs its key. provider_config["auto"] reaches the auto of every method, so
a chain given as a plain list is every method's chain — key it by "search",
"extract", "research", "browser" or "browse" unless every name in it
serves each method you call through auto. The
provider scan prints a suggested search
order in this form, from its own judged accuracy.
This applies only when you name no provider at all. Ask for a provider and a missing key is still an error naming the variable to set — a named backend is never quietly demoted to an anonymous one:
WebRouter(Provider.TAVILY).search("...") # ConfigurationError if TAVILY_API_KEY is unset
How it works
| Piece | Role |
| --- | --- |
| WebRouter / AsyncWebRouter | The facade you call. Holds provider config and routes each call. |
| SearchProvider / ExtractProvider | The contract a backend implements, one per method: _search/_asearch, _extract/_aextract. |
| SearchRequest / ExtractRequest | A normalized call, before any provider sees it. |
| SearchResponse / SearchResult | A normalized answer, plus the provider's untouched payload in raw. |
| ExtractResponse / ExtractedDocument | The same for a page fetch, plus failures for the URLs that did not come back. |
| LiveDataProvider | The contract for a live data backend: _list its catalogue, _get one source, _run it. |
| LiveDataRequest / LiveDataResponse | A normalized run -- a source's name and its own inputs -- and the records it extracted. LiveDataSource describes one, LiveDataSourcePage a page of the catalogue. |
| ResearchProvider | The contract for an agentic backend: _start a run, _fetch where it got to. |
| ResearchRequest / ResearchResponse | A normalized run, and its answer — prose, structure, and the citations behind either. |
| ResearchJob | A run in progress. Polls with back-off, blocks with a timeout, and never fabricates an answer it does not have. |
| BrowseProvider / BrowseJob | The same job shape for a hosted browser agent: BrowseRequest in, BrowseResponse — answer, structure, steps — out. Research and browse share one job core. |
| BrowserProvider | The contract for a browser session backend: _open a session, _release it. |
| BrowserHandle / AsyncBrowserHandle | An open session and its cdp_url, with the page actions, released when the with block ends however it ends. |
Four rules keep the abstraction honest:
Unset stays unset. Fields you leave alone are not sent, so each backend
applies its own defaults rather than inheriting ours. (Three deliberate
exceptions: an extract sends a format even when you named none, markdown
wherever the backend can return it — so the same call answers alike whichever
backend serves it, and document.format is true because it was asked for, not
because a vendor default has not moved. A
browser session opened with no timeout gets ten minutes rather than the
vendor's own limit, because it is billed until something ends it. And local
is headless unless you say otherwise, because there is no vendor default to
keep and a window would open on your screen.)
Nothing is silently dropped. If a provider does not map an option you set —
or cannot return the format you asked for — it says so instead of quietly
doing something weaker:
WebRouter(Provider.LINKUP).search("...", country="US")
UnsupportedParameterError: Provider 'linkup' does not map 'country'. Pass it
through provider_options if the underlying API supports it, or add the
mapping -- see CONTRIBUTING.md.
Nothing is thrown away. Unmapped provider fields land in SearchResult.extra
or ExtractedDocument.extra, and the complete payloads in raw, so
normalization is never lossy.
Nothing fails quietly. An extract over several URLs reports the ones that
did not come back in failures and returns the rest; a call that produced no
document at all raises. An empty answer is never something you have to go
looking for.
Search options
response = router.search(
"nvidia q3 earnings",
max_results=10,
search_depth="fast", # "lite" | "fast" | "deep"
include_domains=["reuters.com", "bloomberg.com"],
exclude_domains=["example.com"],
country="US",
locale="en",
recency="week", # "day" | "week" | "month" | "year"
effort="low", # "low" | "medium" | "high" | "max"
)
Every option except query is optional. Which ones a given provider maps is in
the support table above; setting one it does not map raises rather than being
ignored, and so does a value it cannot serve — a depth level it lacks, a country
off its list — before anything is sent, so auto passes that backend over.
That is the whole surface. It is deliberately small: these are the options that
mean the same thing on every backend, so code written against them keeps working
when you switch providers. search_depth is how much text comes back per
result; effort is how hard the backend looks. The exact field each backend
sends is in its contracts/search/.
Search depth
Three levels, each defined by size, mapped onto each backend's own knobs. A
backend whose longest body is short of a page refuses deep rather than serving
less under its name.
| Depth | What you get | Nimble | Tavily | You.com, You.com MCP | Exa, Exa MCP | WebIQ | Parallel | Perplexity | Keenable | Brave | DeepSeek |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| lite | A snippet, up to about 500 characters | lite | basic, cut to 500 | no extraction, cut to 500 | highlights ≤ 500, cut to 500 | passage ≤ 500 | excerpts ≤ 500, cut to 500 | 100 tokens a page | snippet ≤ 500 | its snippets, cut to 500 | its default |
| fast | A truncated body, a few thousand characters | standard | advanced | highlights | text ≤ 3,000 | passage ≤ 3,000 | excerpts ≤ 3,000 | 1,024 tokens a page | snippet ≤ 3,000 | refused | refused |
| deep | The whole page | standard + full_content | advanced + raw content | full_page | whole text | text | refused | refused | refused | refused | need_content |
Linkup, AgentCore and Parallel MCP do not map it. Where a vendor's size setting
does not bind (Tavily's basic returns up to three chunks of about 500
characters, You.com's snippets can run past 500, Exa's highlights cap,
Parallel's excerpt cap and Perplexity's token budget are soft, DeepSeek's default carries its whole
summary, and Brave has no size setting at all), WebRouter cuts lite to a snippet of about 500
characters itself; the vendor's full text stays in raw.
Unset, Exa asks for the whole text and Exa MCP for highlights, as before.
result.content always holds the fullest text available. Some backends return
the body in the field others use for the snippet; that is normalized away, so
content is the field to read whatever the backend did.
result.description is the provider's snippet **when it returns one as a
separate field**. Not every backend does at every depth — Nimble at fast
returns the page body and no snippet, so description holds that body and is
identical to content. Read description when you want the short form and
have checked your backend supplies one; read content when you want text.
Effort
effort is anchored on each backend's own tiers: medium is what it does
unset, low its fastest tier, max its most thorough, and high between them
or the same as max. A backend with fewer tiers serves several levels with one
setting rather than raising. **Higher effort costs more and takes longer, by a
vendor-specific amount; leave it unset unless needed.**
| Effort | Exa | Exa MCP | Linkup | Parallel | Perplexity | Keenable |
| --- | --- | --- | --- | --- | --- | --- |
| low | lite | lite | flash | turbo | fast | realtime |
| medium | its default | its default | fast | fast | its default | its default |
| high | deep-lite | its default | standard | advanced | its default | pro |
| max | deep | its default | deep | advanced | its default | pro |
Linkup bills deep at ten times its other depths, and Parallel advanced at
five times fast. Parallel refuses low beside a domain filter with a path,
which its turbo mode cannot take. The other backends have no search tier and
refuse effort.
Recency
recency keeps to sources the backend dates within the last day, week, month
or year — by publication date where it has one. A backend that takes a start
date gets today (UTC) minus 1, 7, 30 or 365 days.
| Backend | Sent as |
| --- | --- |
| Nimble | time_range |
| Tavily | time_range with filter_by_published_date, so pages it cannot date are dropped |
| You.com, You.com MCP | freshness |
| Brave | freshness, by the date Brave holds for a page (published or last modified) |
| Perplexity | search_recency_filter |
| Exa, Exa MCP, Linkup, Parallel, AgentCore | a start date (startPublishedDate, fromDate, after_date, publishedDateFilter.from) |
| Keenable | published_after (1d, 7d, 1mo, 1y) |
| DeepSeek | after:YYYY-MM-DD in the query |
WebIQ and Parallel MCP do not map it.
Values a backend refuses
Each of these raises UnsupportedParameterError before anything is sent (a
query too long for its operators raises ProviderError), and auto moves on:
| Backend | Refused |
| --- | --- |
| You.com, You.com MCP | a country off its 36 codes; a locale off its 51-language list (pt or zh alone; give pt-BR, zh-TW); both domain lists at once |
| Brave | a country off its 37 codes; a locale off the same language list; a query past 600 characters or 75 words with its site: operators |
| Tavily | a country off the 166 it names, or with a topic other than general (country is a ranking boost there; locale is a filter) |
| Parallel | a country off its 37 codes; both domain lists at once; more than 200 domains |
| Perplexity | both domain lists at once; more than 20 domains; a locale that is not a bare two-letter language (en, not en-US) |
| WebIQ | a country or locale that is not two letters (a regional locale sends its language); a query past 1,000 characters with its site: operators |
| Keenable | more than one include_domains entry |
| DeepSeek | a query past 128 characters with its operators |
| Brave, WebIQ, DeepSeek, You.com MCP | a domain filter that is not a host name (optionally with a path), since it goes into the query as site: (for You.com MCP, include_domains only; exclude_domains is a tool argument) |
Extracting page contents
extract() is the other half: give it URLs you already have and it returns the
pages, through whichever backend you point it at.
response = router.extract(
["https://example.com", "https://www.iana.org/domains/reserved"],
format="markdown", # "markdown" | "html" | "text"
)
for document in response.documents:
print(document.url, document.format, len(document.content))
One URL or many — router.extract("https://example.com") is the same call, and
the answer has the same shape. Backends that take a single URL per request are
fanned out for you, concurrently on AsyncWebRouter, and the documents come back
in the order you asked for them.
Formats
format is the extract equivalent of search_depth: three names for what the
page should come back as, mapped onto each backend's own vocabulary.
| Format | What you get | Nimble | Tavily | Firecrawl | Oxylabs | Bright Data | Jina | Linkup | Browserbase |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| markdown | Structure kept, markup dropped | formats: [markdown] | format: markdown | formats: [markdown] | markdown: true | data_format: markdown | X-Return-Format: markdown | always returned | format: markdown |
| html | The page's HTML as fetched | formats: [html] | — | formats: [html] | default (type: raw) | default | X-Return-Format: html | includeRawContent: true | format: raw (the body as served) |
| text | Visible text, no markup | — | format: text | — | — | — | X-Return-Format: text | — | — |
Exa is left out of the table because it has one row to give: its Contents API
returns text and nothing else, asked for as text: true. WebIQ is left out
too: Browse takes contentFormat: markdown or text, and its html is a
simplified markup rather than the page as fetched, so html
... (README truncated for length)