How it works
Routing, retries, timeouts, and capabilities.
- Validate the URL (
http/httpsonly). - Select providers that advertise the required capability (
scrape,search,crawl,extract,js). - Order them by
strategy:priority(config order) orcost(cheaper first). - Execute with an
AbortSignaltimeout. Retryable errors (429, 5xx, timeout) retry on the same provider, then fail over. Auth errors (401/403) do not retry. - Return a unified result, or
AllProvidersFailedError/CapabilityError.
Local Cheerio is the only adapter that sanitizes HTML itself. Cloud adapters return the vendor's markdown.
There is no fake Browserbase page content and no crawl that pretends POST /crawl already contains pages.