# How it works (https://www.scrape-sdk.com/docs/how-it-works)

Canonical: https://www.scrape-sdk.com/docs/how-it-works
Markdown: https://www.scrape-sdk.com/docs/how-it-works.md



1. **Validate** the URL (`http` / `https` only).
2. **Prefer `/llms.txt`** on a site root or `/docs` path (skip article URLs). A 1.5s probe; HTML scrape if it misses.
3. **Select** providers that advertise the required capability (`scrape`, `search`, `crawl`, `extract`, `js`).
4. **Order** them by `strategy`: `priority` (config order) or `cost` (cheaper first).
5. **Execute** with an `AbortSignal` timeout. Retryable errors (429, 5xx, timeout) retry on the same provider, then fail over. Auth errors (401/403) do not retry.
6. **Return** a unified result with `provider`, `latencyMs`, and `failedOverFrom` when a hop happened — or `AllProvidersFailedError` / `CapabilityError`.

Local Cheerio is the only adapter that sanitizes HTML itself. Cloud adapters return the vendor's markdown.

There is no fake Browserbase page content and no crawl that pretends POST `/crawl` already contains pages.