> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape

> Scrape 1 to 3 URLs and get the pages as Markdown, or the data you describe as JSON. Real browsers, actions, schemas, screenshots, Stealth Mode and documents.

Scrape reads 1 to 3 pages in a real browser and returns them as Markdown, or extracts the data you describe. With several URLs, the answer combines what's on all of them. For a separate result per URL, use [Batch scrape](https://docs.spidra.io/batch-scrape.md); for a whole site, use [Crawl](https://docs.spidra.io/crawl.md).

## Quick example

Get the title and price of every book on a page:

```bash curl theme={null}
curl -X POST https://api.spidra.io/api/scrape \
  -H "Authorization: Bearer $SPIDRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [{ "url": "https://books.toscrape.com" }],
    "prompt": "Return the title and price of every book on the page",
    "output": "json"
  }'
```

```typescript Node.js theme={null}
import { SpidraClient } from "spidra";

const spidra = new SpidraClient({ apiKey: process.env.SPIDRA_API_KEY });

const result = await spidra.scrape({
  urls: [{ url: "https://books.toscrape.com" }],
  prompt: "Return the title and price of every book on the page",
  output: "json",
});
```

```python Python theme={null}
import os
from spidra import Spidra

client = Spidra(api_key=os.environ["SPIDRA_API_KEY"])

result = client.scrape(
    "https://books.toscrape.com",
    prompt="Return the title and price of every book on the page",
    output="json",
)
```

The API answers with a `jobId`. Poll [GET /scrape/\{jobId}](https://docs.spidra.io/api-reference/scraping/scrape-status.md) until `status` is `completed`; the SDKs do this for you and return the result:

```json theme={null}
{
  "status": "completed",
  "result": {
    "content": [
      { "title": "A Light in the Attic", "price": "£51.77" },
      { "title": "Tipping the Velvet", "price": "£53.74" }
    ],
    "data": [
      { "url": "https://books.toscrape.com", "title": "All products | Books to Scrape - Sandbox", "markdownContent": "...", "success": true }
    ],
    "stats": { "durationMs": 37458, "inputTokens": 2628, "outputTokens": 418, "totalTokens": 3046 }
  }
}
```

The list had all 20 books (shortened here). It cost 3 credits: 1 for the page and 2 for the AI.

## What you get back

* **`result.content`** is the answer. Without a prompt it's the page as Markdown; with a prompt or schema it's what you asked for.
* **`result.data`** has one entry per URL, in the order you sent them, with the page's `markdownContent`, `title` and `success`. Links from [supported sites](https://docs.spidra.io/sites/overview.md) also have `structuredData` with the site's own fields.
* **`result.stats`** has the run time and the AI tokens used.
* **`result.screenshots`** has image links when you ask for screenshots.

## Raw Markdown, a prompt, or a schema

| You send | `content` | Cost |
| - | - | - |
| Only the URL | The page as clean Markdown | 1 credit |
| `prompt` | What you described, in a shape the AI picks | 1 credit plus AI |
| `prompt` and `schema` | Exactly your fields and types, `null` for anything missing | 1 credit plus AI |

Without a schema, field names and types can change between runs: prices come back as `"£51.77"` here. With one, you get the same shape every time:

```json theme={null}
{
  "urls": [{ "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }],
  "prompt": "Extract the book details",
  "schema": {
    "type": "object",
    "required": ["title", "price"],
    "properties": {
      "title": { "type": "string" },
      "price": { "type": "number" },
      "currency": { "type": ["string", "null"] },
      "in_stock": { "type": ["boolean", "null"] },
      "rating": { "type": ["string", "null"] }
    }
  }
}
```

```json theme={null}
{ "title": "A Light in the Attic", "price": 51.77, "currency": "£", "in_stock": true, "rating": null }
```

`rating` is `null` because that page shows its stars as an image, not text. See [Structured output](https://docs.spidra.io/options/structured-output.md) for writing schemas.

## Options

| Option | What it does |
| - | - |
| `prompt` | What to extract, in plain words. |
| `output` | `"markdown"` (default) or `"json"`. |
| `schema` | A JSON Schema for the result. Forces JSON output. |
| `actions` (per URL) | Click, type, scroll, wait or loop over items before the page is read. See [Browser actions](https://docs.spidra.io/options/browser-actions.md). |
| `extractContentOnly` | Drop menus, headers, footers and sidebars before reading the page. |
| `screenshot`, `fullPageScreenshot` | Capture the visible page, or the whole page. |
| `scrapeMode` | `"fast"` reads the page over HTTP without a browser. Quicker for static pages; actions and screenshots are off. |
| `useProxy`, `proxyCountry` | Stealth Mode: send the request through residential proxies, in a country such as `"us"` or `"de"`. See [Proxies and CAPTCHAs](https://docs.spidra.io/options/proxies-and-captchas.md). |
| `cookies` | Read pages behind your own login. See [Logged-in pages](https://docs.spidra.io/options/logged-in-pages.md). |

Every field is in the [API reference](https://docs.spidra.io/api-reference/scraping/scrape.md).

## Several URLs

Send up to 3 URLs to answer one question from several pages, such as comparing two products:

```json theme={null}
{
  "urls": [
    { "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" },
    { "url": "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html" }
  ],
  "prompt": "Which of these two books is cheaper, and by how much?"
}
```

The AI reads both pages and gives one answer in `content`. Each page's Markdown is still in `data`. Without a prompt, `content` joins the pages, each starting with a `Content from: <url>` line.

## Documents

Point Scrape at a PDF, Word, Excel, PowerPoint, OpenDocument, RTF, EPUB or CSV link and you get the document as Markdown. Scanned pages are read with AI. For uploads, page ranges, a cost estimate first, or many documents, use [Parse](https://docs.spidra.io/parse.md). See [Documents by URL](https://docs.spidra.io/options/documents.md).

## How it works

* Each page opens in a real browser, so pages built with JavaScript work. Cookie banners are dismissed and CAPTCHAs are solved automatically.
* A page that can't be read fails the job with a plain `error`, such as "the page answered HTTP 404", and costs nothing. So does a page that comes back blank or behind a sign-in wall.
* Links from [supported sites](https://docs.spidra.io/sites/overview.md) are read directly, without a browser, and come back with the site's own fields.
* Sending the same scrape twice within a few seconds returns the same job.

## Cost

1 credit per URL, plus AI credits when you use a prompt or schema (usually 1 or 2 per page, more on long pages), plus 5 per CAPTCHA solved. Screenshots, actions and Fast Mode cost nothing extra. Stealth Mode uses proxy bandwidth. Failed jobs are free. See [How credits work](https://docs.spidra.io/billing/how-credits-work.md).

## In the dashboard

Open **Scrape** in the sidebar. The form has the same options: up to 3 URLs (each with **Add action** for browser actions), **Extraction instruction**, **Output** (Markdown, JSON or Screenshot), **Define schema**, **Stealth mode**, **Extract content only**, **Authentication** for cookies, and the **Fast mode** switch. **Code** shows the request for your settings in curl, JavaScript or Python, and **Save as preset** keeps the setup for later.

![The Scrape page with the form on the left and the output on the right](https://mintcdn.com/spidra/gLoAKV-50tHrEESG/images/scrape-playground.webp?fit=max\&auto=format\&n=gLoAKV-50tHrEESG\&q=85\&s=3d313316c740227b4b38c3e01a91bcd4)

## When the result isn't what you expected

* **`content` is empty or `{}`.** The data may load after the page opens. Add a `scroll` or `wait` [action](https://docs.spidra.io/options/browser-actions.md), or take a screenshot to see what Spidra saw.
* **A field is `null`.** It isn't in the page's text. Check the page, then reword the prompt or the field.
* **The job failed.** `error` says why. You weren't charged.
* **The site blocks you.** Turn on Stealth Mode, and try a country where the site is popular.

## Next

> **[Batch scrape](https://docs.spidra.io/batch-scrape.md)**
>
> Up to 50 URLs, one result each.

> **[Structured output](https://docs.spidra.io/options/structured-output.md)**
>
> Schemas for exact fields and types.

> **[Browser actions](https://docs.spidra.io/options/browser-actions.md)**
>
> Click, scroll and loop before reading.

> **[API reference](https://docs.spidra.io/api-reference/scraping/scrape.md)**
>
> Every field and response.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.