Scrape SDK
API Reference

createScrapeClient

Client factory, fromEnv, and methods.

View Markdown
createScrapeClient({
  providers?: ScrapeProvider[];
  provider?: ScrapeProvider;          // alias for a single primary
  fallback?: ScrapeProvider | ScrapeProvider[];
  timeoutMs?: number;                 // default 30000, abortable
  retries?: number;                   // extra attempts per provider on retryable errors; default 1
  strategy?: "priority" | "cost";
  cache?: boolean | { ttlMs?: number; maxEntries?: number };
  onFailover?: (error: Error, from: string, to: string) => void;
})
fromEnv() // builds providers from process.env keys, always ends with jina + local

Methods

  • scrape(url, options?)maxChars truncates markdown and sets truncated / charCount
  • scrapeMany(urls, { concurrency?, ...scrapeOptions })
  • search(query, { limit?, includeAnswer?, includeRawContent? })
  • map(url, { limit?, search? }) — URL list without page bodies (Firecrawl)
  • crawl(url, { limit?, maxDepth?, pollIntervalMs?, maxChars?, ... })
  • extract(url, { schema, prompt? })
  • supports(capability)"scrape" | "search" | "crawl" | "extract" | "js" | "map"
  • listProviders()

On this page