Endpoint
X-API-Key header. See Authentication.
Request
Parameters
URLs to fetch and extract. Maximum 10 URLs per request.All URLs must use
http or https. Private IP addresses, localhost, and cloud metadata endpoints are rejected.An optional short statement of why you are fetching these URLs — the underlying goal or task
the content will be used for. Supplying the intent behind a fetch gives us additional signal to
further inform and deliver better-quality results. When omitted, fetch behaves exactly as before.
When supplied, the value must be non-empty (whitespace-only values are rejected). Maximum 2000
characters. For example,
Compare pricing tiers across vendors for a procurement report.Output format for the
text field in each result. One of:html— semantic HTMLmarkdown— clean Markdown, recommended for LLMs (default)json— structured document tree
When
true, include all <img src> URLs found on the page in the image_links field.When
true, include the src URLs of <iframe> elements found on the page in the iframe_links field.Use this when a page embeds its main content in a frame (job boards, booking widgets, document viewers): fetch the returned URL to read the embedded content. Iframes that a page injects with JavaScript are only present when the page was rendered in a browser.Cache freshness tolerance in seconds.
- Omit
ttlto accept any cached entry. - Set
ttlto0when you want a live fetch. - Set
ttlto a positive integer to accept cached entries younger than that many seconds.
Per-URL wall-clock timeout budget in milliseconds. Must be between
1 and 110000.If a URL exceeds this budget, that URL returns a timeout error in errors[] while other URLs in the same request can still complete.An ETag you saved from a prior fetch of this URL, forwarded verbatim as the
If-None-Match header on the origin request. Single URL only — combining with a batch of URLs returns a 400.A
Last-Modified value you saved from a prior fetch of this URL, forwarded verbatim as the If-Modified-Since header on the origin request. Single URL only — combining with a batch of URLs returns a 400.When
true, include etag and last_modified on each result so you can store them and replay them on your next fetch of the same URL. Works with a single URL or a batch.Conditional requests are a stateless pass-through — Fetch does not store
etag or last_modified for you. Save the values returned by include_etag_and_last_modified and replay them as if_none_match / if_modified_since on your next request to detect whether the page changed.An array of CSS selectors (1–20 entries, each 1–1000 characters) that scopes extracted content (
text, links, image_links) to elements matching any entry, concatenated in document order. Tag selectors cover semantic sections (main, article, nav); entries may themselves use CSS comma-grouping ("main, footer"). Standard CSS only — not XPath, Playwright text= locators, or glob patterns such as *specificat* (attribute matching like [class*="price"] is fine). Ids that start with a digit or contain dots are not valid #id selectors; use the attribute form instead ([id="48599"]).Selected content is returned verbatim, converted to the requested format (scripts and styles stripped) — automatic boilerplate removal is bypassed. Page-level metadata (title, description, language, author, published_date) still comes from the full document.If some entries match and others don’t, the URL still succeeds with the matched content and the misses are listed in the result’s unmatched_selectors. If no entry matches anything, that URL fails with the per-URL error code selector_not_matched — there is no silent full-page fallback — and the error carries unmatched_selectors plus candidate_selectors retry hints. Invalid CSS selector syntax is rejected with a 422.An array of CSS selectors (1–20 entries, each 1–1000 characters) for elements to remove before extraction. Applied before
include_selectors scopes what remains, so it also prunes inside selected regions. Entries that match nothing are a no-op, never an error. Same syntax rules as include_selectors: standard CSS only, attribute form for digit-leading or dotted ids. Invalid CSS selector syntax is rejected with a 422.Selectors are applied after the page is fetched — caching and routing are unchanged. URLs that resolve to direct PDF or CSV downloads have no HTML to scope, so sending
include_selectors or exclude_selectors for them fails that URL with selector_unsupported.Ask a question and get back the passages from each page that best answer it — ranked, and quoted word-for-word from the page. Requires
format: "markdown". Sub-fields below.The question to answer from each page. Include the entity or topic in the query (e.g. “who founded Acme Corp”, not “who founded this company”) — retrieval works on vocabulary overlap with the page, and keyword-poor queries return empty results far more often.
Soft total-character budget across a URL’s passages, between
100 and 20000. Passages are added until the budget is reached, but the top-ranked passage is always returned even if it alone exceeds the budget.By default a highlights request returns just the passages — the result’s
text field is null. Set true to also receive the full page markdown in text.Beta: Highlights is in beta and is enabled per-account — join our Beta program to enable it for your account. Requests that include
highlights from an account that isn’t enabled return 403 FORBIDDEN; requests without the field are unaffected.Highlights are computed for standard web pages. PDFs and other non-HTML documents are still fetched and their text returned, but
highlights is not computed for them — the field is absent on those results.Response
results[]
One entry per successfully fetched URL.
Meta description, preferring
og:description over <meta name="description">. null if not found.Extracted page content. Format depends on the
format request parameter:stringwhenformatis"html"or"markdown"object(document tree) whenformatis"json"nullon a highlights request, unlesshighlights.include_full_page_text: true
All
<a href> URLs on the page, resolved to absolute URLs. Only present when links: true was requested.All
<img src> URLs on the page, resolved to absolute URLs. Only present when image_links: true was requested.<iframe src> URLs on the page, resolved to absolute URLs and deduplicated, in document order. Only present when iframe_links: true was requested.Present and
true only when the origin returned 304 Not Modified for your if_none_match or if_modified_since header — the content is unchanged and you can skip re-processing. Omitted otherwise.The origin’s current ETag validator. Save it and replay it as
if_none_match on your next fetch of this URL. Present only when the origin sent one and you set include_etag_and_last_modified; omitted otherwise.The origin’s current Last-Modified validator. Save it and replay it as
if_modified_since on your next fetch of this URL. Present only when the origin sent one and you set include_etag_and_last_modified; omitted otherwise.Partial-miss report for
include_selectors: the entries that matched no elements on this page. Present only when some (but not all) entries matched — the result still carries the matched content. Omitted when every entry matched. A total miss is reported as a selector_not_matched entry in errors[] instead.Ranked verbatim passages answering the request’s
highlights.query, best first. Each entry is { "text": string, "rank": integer }, with rank starting at 1. Present only on highlights requests.An empty array means the page was fetched and analyzed and does not answer the query — a deliberate, calibrated signal, not a failure. A missing field on a highlights request means highlights could not be computed for this URL (rare and transient); the fetch itself still succeeded, and a retry is safe.Output format used for the
text field. Echoes the request format parameter ("markdown", "html", or "json").Fields that could not be extracted (
title, description, language, author, published_date) are returned as null.errors[]
One entry per URL that could not be fetched. Always present, may be empty. Per-URL failures do not affect the rest of the batch.
Structured error code identifying the failure type. One of:
target_http_error, page_not_found, target_unreachable, timeout, bot_blocked, empty_content, login_required, content_too_large, invalid_url, invalid_redirect_url, proxy_error, conditional_unsupported, selector_not_matched, selector_unsupported.Upstream HTTP status code. Present when
error is target_http_error or page_not_found.The
include_selectors entries that matched no elements on the fetched page. Present only when error is selector_not_matched.Cheap retry hints (max 10), present only when
error is selector_not_matched: landmark tags found on the page (e.g. main, article, nav) plus #id selectors of content-heavy elements. Agents can retry the fetch with include_selectors drawn from this list.SDK Methods
Error Codes
HTTP-level errors apply to the entire request.
Per-URL errors appear in
errors[] alongside a 200 response. The error field is one of these codes:
Per-URL fetch failures are not HTTP errors. They appear as entries in
errors[] alongside a 200 response.Each URL has a 110-second backend timeout. If the page doesn’t respond within 110 seconds, that URL returns a
timeout error in errors[] while the rest of the batch continues. Requests are also subject to a 120-second CDN ceiling for the full batch. Set your client-side timeout to at least 150 seconds to receive CDN timeout errors cleanly.Supported Content Types
Usage Endpoint
Retrieve a paginated history of your fetch operations. An organization workflow key lists fetches made by any of its team’s workflow keys, never its creator’s own; see Who sees what.X-API-Key header. See Authentication.
Query Parameters
Response
items[]
Length of the extracted text content in characters. The full text is not included in usage responses.
Present and
true only when that fetch was a conditional request that returned 304 Not Modified. Omitted otherwise.The origin’s ETag validator captured at fetch time. Present only when
include_etag_and_last_modified was set on that request and the origin returned one; omitted otherwise.The origin’s Last-Modified validator captured at fetch time. Present only when
include_etag_and_last_modified was set on that request and the origin returned one; omitted otherwise.Error Codes
Rate Limits
Limits apply per person, or per organization when you work for one, measured in URLs per minute across all requests. Personal accounts default to 150 URLs / minute; an organization’s limit comes from its contract. When the limit is exceeded, the API returnsHTTP 429.
Billing
Fetch is free up to 1,000 URLs per day, counted per successfully fetched URL; the allowance resets at 00:00 UTC. Beyond it, requests need a positive wallet balance: with a balance of zero or below the API returnsHTTP 402 with code INSUFFICIENT_CREDITS until you add funds or the allowance resets. After a top-up, allow a few seconds for the balance to apply. A product that does not charge the wallet (beta) returns FREE_ALLOWANCE_EXHAUSTED instead once its allowance is used; only the daily reset lifts it.
Related
Fetch Overview
First request, response shape, and product routing
Authentication
API key setup and troubleshooting
Error Codes
Full list of API error codes