> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Batch scrape

> Scrape up to 50 URLs in one request and get a separate result for each, with per-item cost, free failures and one-call retries.

Batch scrape takes up to 50 URLs and gives each one its own result. Use it when you have a list of pages with the same kind of data: products, articles, listings, profiles. Each URL succeeds or fails on its own, and failed ones can be retried in one call.

Don't have the URLs yet? [Map](https://docs.spidra.io/map.md) a site to find them, or [Crawl](https://docs.spidra.io/crawl.md) it.

## Quick example

Get the title and price from two book pages, plus one URL that doesn't exist:

```bash curl theme={null}
curl -X POST https://api.spidra.io/api/batch/scrape \
  -H "Authorization: Bearer $SPIDRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
      "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
      "https://this-domain-does-not-exist-spidra-test.invalid"
    ],
    "prompt": "Extract the book details",
    "schema": {
      "type": "object",
      "properties": { "title": { "type": "string" }, "price": { "type": "number" } }
    },
    "wait": true
  }'
```

```typescript Node.js theme={null}
const batch = await spidra.batchScrape({
  urls: [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
    "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
  ],
  prompt: "Extract the book details",
  schema: { type: "object", properties: { title: { type: "string" }, price: { type: "number" } } },
});

for (const item of batch.items) console.log(item.status, item.result);
// completed { title: 'A Light in the Attic', price: 51.77 }
// completed { title: 'Tipping the Velvet', price: 53.74 }
```

```python Python theme={null}
batch = client.batch_scrape(
    [
        "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
        "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
    ],
    prompt="Extract the book details",
    schema={"type": "object", "properties": {"title": {"type": "string"}, "price": {"type": "number"}}},
)

for item in batch.items:
    print(item.status, item.result)
# completed {'title': 'A Light in the Attic', 'price': 51.77}
# completed {'title': 'Tipping the Velvet', 'price': 53.74}
```

With `"wait": true` and up to 10 URLs, the result comes back in the same request if it finishes within 45 seconds. This one took 8 seconds (items shortened):

```json theme={null}
{
  "batchId": "38ae74ab-3a7f-421f-9d02-50c6b48d8c49",
  "status": "completed",
  "totalUrls": 3,
  "completedCount": 2,
  "failedCount": 1,
  "items": [
    {
      "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
      "status": "completed",
      "result": { "title": "A Light in the Attic", "price": 51.77 },
      "error": null,
      "creditsUsed": 2
    },
    {
      "url": "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
      "status": "completed",
      "result": { "title": "Tipping the Velvet", "price": 53.74 },
      "error": null,
      "creditsUsed": 2
    },
    {
      "url": "https://this-domain-does-not-exist-spidra-test.invalid",
      "status": "failed",
      "result": null,
      "error": "Failed to scrape https://this-domain-does-not-exist-spidra-test.invalid: the page could not be reached (ERR_TUNNEL_CONNECTION_FAILED)",
      "creditsUsed": 0
    }
  ],
  "createdAt": "2026-10-25T03:09:37.000Z",
  "finishedAt": "2026-10-25T03:09:45.000Z"
}
```

Without `wait`, or with more than 10 URLs, you get `202` with a `batchId` and poll [GET /batch/scrape/\{batchId}](https://docs.spidra.io/api-reference/scraping/batch-scrape-status.md). The SDKs poll for you.

## What you get back

* **`status`** is `completed` once every URL is done, **even if some failed**. Check `failedCount` and each item's `status`.
* **`items`** has one entry per URL with its own `status`, `result`, `error` and `creditsUsed`. With a prompt or schema, `result` is the parsed data; without one, it's the page as Markdown.
* Items from [supported sites](https://docs.spidra.io/sites/overview.md) also have `structuredData` with the site's own fields.
* `completedCount` and `failedCount` count up while it runs, so you can show progress.

## Options

The same options as [Scrape](https://docs.spidra.io/scrape.md), applied to every URL: `prompt`, `output`, `schema`, `extractContentOnly`, `screenshot`, `fullPageScreenshot`, `scrapeMode`, `useProxy`, `proxyCountry` and `cookies`. Plus `wait`.

`urls` takes plain strings or `{ "url": "..." }` objects. Per-URL browser actions aren't available in a batch; use [Scrape](https://docs.spidra.io/scrape.md) for pages that need them.

## Retry the URLs that failed

```bash theme={null}
curl -X POST https://api.spidra.io/api/batch/scrape/38ae74ab-3a7f-421f-9d02-50c6b48d8c49/retry \
  -H "Authorization: Bearer $SPIDRA_API_KEY"
```

```json theme={null}
{ "status": "queued", "retriedCount": 1 }
```

Only failed items run again, with the batch's original options. Cookies are never stored, so send `cookies` in the retry body again if the batch used them.

## Cancel

`DELETE /batch/scrape/{batchId}` stops the items that haven't started and gives back the credits held for them. The response says how many items it stopped (`cancelledItems`) and how many credits came back (`creditsRefunded`). Items already done keep their results.

## Cost

Each URL costs what a scrape of that URL costs: 1 credit, plus AI credits with a prompt or schema (2 per book page above). Failed items cost nothing. See [How credits work](https://docs.spidra.io/billing/how-credits-work.md).

## Limits

* 50 URLs per batch.
* 20 batches a minute, and up to 50 of your batch URLs unfinished at once (counted together with scrapes).
* `wait` only for 10 URLs or fewer.

## In the dashboard

Batches are started from the API, an SDK, the [MCP server](https://docs.spidra.io/sdks/mcp.md) or a [template](https://docs.spidra.io/templates.md). Every batch shows in [Logs](https://docs.spidra.io/dashboard/logs.md) under **Batch**, where you can open it to see each URL's result, status and cost.

## Next

> **[Map](https://docs.spidra.io/map.md)**
>
> Find the URLs to batch.

> **[Scrape a list of URLs to CSV](https://docs.spidra.io/guides/urls-to-csv.md)**
>
> A complete script.

> **[Structured output](https://docs.spidra.io/options/structured-output.md)**
>
> Schemas for exact fields.

> **[API reference](https://docs.spidra.io/api-reference/scraping/batch-scrape.md)**
>
> Every field and response.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.