Web scraping API
Turn any page into data. Even the ones that fight back.
Headless browsers, rotating datacenter and ISP exits, and an extraction engine, behind one URL. Send a link, pick a shape, get it back. Blocked attempts are never charged.
1,000 free credits, then 250 a month. No card, price locked for life. Migrating from ScrapingBee?
1,000 free credits, then 250 a month no card 1 credit raw, 5 rendered, 75 stealth failed calls cost 0 5 MB stealth ceiling, 25 MB otherwise price locked for life
-
No card, ever
1,000 credits on the free tier, then 250 more every month rather than a trial that runs out. An email address is the whole signup.
-
Failed calls are free
Billing happens on success and nowhere else. There is no refund step because nothing was charged in the first place.
-
Price locked for life
Sign up at today's price and keep it for as long as the payment method stays active and the subscription does not lapse.
-
Safe by default
robots.txthonored on every request, and every host resolved, validated and pinned before a byte is sent. Both re-checked on each redirect. Opting out of robots needs an accepted acceptable use policy.
Try it on a real page
This runs an actual scrape against the URL you give it, on the same engine the API uses. No key needed for the basics, and the page itself is not stored.
Sign in or create a free account for every option
-
The page as a browser saw it
Rendered in a browser, so content that only exists after hydration is there. -
Or clean text and markdown
Navigation, cookie banners and footers stripped. -
Or JSON you defined
CSS or XPath selectors inextract_rules. -
The target's real status
Plus the resolved URL and the exact credit cost. -
Nothing, if it failed
A failed fetch is never billed.
Three steps to your first scrape
No sales call, no onboarding, no card. The whole path from stranger to working integration is an email address and one HTTP request.
Confirm an email
Enter an address, type the six-digit code we send it, and the key is on screen. That is the entire signup.
Call the endpoint
One GET against /v1/ with a url. Everything
else has a sensible default, and every default matches ScrapingBee's.
Take the shape you want
Raw HTML, cleaned text, markdown, a screenshot, or JSON you defined with
extract_rules. Parsing on your side is optional.
What it does
Rendered or raw
A browser when you need hydrated content, a plain HTTP fetch when you do not. wait, wait_for and wait_browser control how long to settle.
Structured extraction
extract_rules turns CSS or XPath selectors into JSON, including nested objects, lists and whole tables. No parsing on your side.
Browser actions
js_scenario clicks, fills, scrolls and waits before the page is captured, so content behind an interaction is reachable.
Text and markdown
return_page_text and return_page_markdown hand back clean content for indexing or for a model, with menus, headers and footers stripped.
Rotating proxy pools
premium_proxy for static ISP proxies, stealth_proxy for ISP exits in a hardened browser, selected per request. country_code targets a region, session_id pins an exit IP.
Automatic escalation
mode=auto retries a blocked page on progressively more capable routes and bills only the one that worked, bounded by max_cost.
Screenshots
screenshot and screenshot_full_page are captured in the same page load as the HTML and come back as a link, at no extra credit cost.
Charged on success only
A failed fetch costs nothing. There is no refund step because a failure is never billed in the first place.
mode=auto
Let it find the route that works
Hard targets do not need a bigger budget, they need the right exit. Auto mode tries the same five configurations as ScrapingBee's, cheapest first (1, 5, 10, 25 and 75 credits), and climbs only when the target refuses: a 403, a 429, a 5xx, a timeout, a failed render, or an anti-bot challenge or CAPTCHA page, whatever its status.
A 404 is a real answer, so it is never retried. Routes that failed cost nothing, so
you are billed once, for the route that worked. max_cost is the ceiling,
and Wf-Auto-Attempts tells you exactly which configurations were tried.
Wf-Auto-Attempts: raw,render_js,premium_proxy
What people point it at
The same endpoint, shaped by the parameters. None of these need a different product.
Price and stock monitoring
Retail pages are rendered and often defended. render_js with extract_rules returns just the price, the currency and the availability as JSON, on a schedule.
Search and listing results
The Search API returns ranked Google, Bing and Yahoo results and their ads as JSON. On other sites, js_scenario clicks through, extract_rules with type: list returns the rows.
Content for models
return_page_markdown gives clean prose with the navigation, cookie banners and footers stripped, which is what a retrieval pipeline wants.
Lead and directory data
Whole tables come back with table_json, keyed by their own headers, without writing a parser for each site's markup.
Compliance and archival
screenshot_full_page captures how a page looked, and the raw HTML is kept alongside it, so an assertion about a page is evidenced.
Uptime and change detection
transparent_status_code passes the target's own status through untouched, so a monitor sees what a browser would have seen.
Search engine results as JSON
Google, Bing and Yahoo results from one GET request, in the shape of ScrapingBee's Google Search API: ranked organic results with their destination URLs, the ads above and below them, related searches and questions. 10 credits a search, and a failed search costs nothing.
Google Search API
Organic results with the destination URL of every link, top and bottom ads, People also ask questions and related searches.
Bing Search API
Organic results, ads, related searches and People also ask questions. A light request is a plain fetch that answers in about half a second.
Yahoo Search API
Organic results with their sitelinks, top and bottom sponsored results, and related searches.
Run a search in the browser and download the results as JSON or CSV from Scan, or read the Search API overview. Paid plans turn search on in the key portal; free accounts can request it there.
A drop-in change
Before
curl "https://app.scrapingbee.com/api/v1/\
?api_key=$KEY\
&url=https%3A%2F%2Fexample.com\
&render_js=true"After
curl "https://api.webfetch.io/v1/\
?api_key=$KEY\
&url=https%3A%2F%2Fexample.com\
&render_js=true"
Every parameter keeps its name and its default. render_js defaults to
true, block_resources to true, timeout to 140000 ms. Response
headers carry the same values under a Wf- prefix instead of
Spb-.
Same credits, a lower bill
Credit costs and monthly allowances match ScrapingBee's exactly, so nothing about your usage math changes if you're coming from there. Every plan below is 38 to 40% less.
| Plan | Credits per month | Webfetch | ScrapingBee |
|---|---|---|---|
| Starter | 75,000 | $11.99 | No plan this size |
| Pro | 250,000 | $29.99 | $49 (Freelance) |
| Scale | 1,000,000 | $59.99 | $99 (Startup) |
| Business | 3,000,000 | $149.99 | $249 (Business) |
| Business+ | 8,000,000 | $359.99 | $599 (Business+) |
A free account starts with 1,000 credits and keeps going with 250 more every month, against ScrapingBee's one-time 1,000-credit trial. Need dedicated hardware with no cap on datacenter credits? Custom starts at $499. Full pricing.
Network
Real exits in real countries
Requests run on dedicated fetch nodes and leave through our own proxy pools, never from
the server that answers the API. Target a country with country_code, and a
country we cannot exit from is refused rather than quietly served from somewhere else.
Fetch nodes
- France
- United States
Datacenter exits
- Argentina
- Australia
- China
- Colombia
- Germany
- India
- Indonesia
- Japan
- Mexico
- Saudi Arabia
- South Africa
- Turkey
- United Kingdom
- United States
Static ISP exits
- Australia
- Brazil
- Canada
- Germany
- Italy
- South Korea
- Spain
- United Kingdom
- United States
Questions people ask first
Is the free account a trial?
No, it keeps going. You start with 1,000 credits and get 250 more on the 1st of every month, banking up to 1,000, for as long as the account exists. ScrapingBee's 1,000 credits are a one-time trial. No card is asked for.
What does a failed request cost?
Nothing. A request is billed only when it succeeds, and the credit figure is written on the request row at that moment. There is no refund process because there is nothing to refund.
What does price locked for life mean?
The price you sign up at is the price you keep, for as long as the payment method stays active and the subscription does not lapse. If we raise prices later, your invoice does not move.
How many credits does a request cost?
One for a raw fetch, five with JavaScript rendering, ten through the premium proxy pool, twenty-five for premium plus rendering, and seventy-five through the stealth pool. Exactly ScrapingBee's table, so your per-call arithmetic does not change.
Is there a limit on how much a single scrape can download?
Yes. Five megabytes through the stealth pool and twenty-five everywhere else, measured on the wire rather than on the returned document, so the assets a page loads count toward it. A page that reaches the ceiling still returns what was captured, billed normally, marked with Wf-Truncated. Ordinary pages are nowhere near it.
Do you respect robots.txt?
By default, yes, on every request. Opting out is available on paid plans only: it is turned on for the key in the key portal, which records your acceptance of the acceptable use policy, and every request that opts out is logged.
Why are some sites, like banks or payment processors, refused?
Our proxy network will not connect to some categories of sites, such as banks, payment processors and some government services. A request for one fails at once with 403 target_restricted and costs nothing. It applies on every plan and every pool, so no parameter changes it. The network page has the details.
Do I have to change my code to migrate from ScrapingBee?
No. Change the base URL to https://api.webfetch.io/v1/ and swap the key. Every parameter keeps its name, its type and its default, the credit costs are identical, and response headers carry the same values under a Wf- prefix instead of Spb-.
Can I scrape a site that blocks datacenter traffic?
Set stealth_proxy=true for ISP exits in a browser hardened against headless detection, or let mode=auto escalate for you and pay only for the route that worked.
Ready to fetch something?
1,000 free credits, then 250 a month. No card, no sales call. If it does not do what you need, you have spent an email address.