> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Core concepts

> How Spidra works in one page: the tools, how a request becomes a job and a result, what comes back, what costs credits and what's free.

## The tools

| Tool | Input | Output | Answers |
| - | - | - | - |
| [Scrape](https://docs.spidra.io/scrape.md) | 1 to 3 URLs | One combined result: the pages as Markdown, or the data you describe | As a job |
| [Batch scrape](https://docs.spidra.io/batch-scrape.md) | Up to 50 URLs | One result per URL | As a job, or straight away with `wait` for up to 10 URLs |
| [Crawl](https://docs.spidra.io/crawl.md) | A start URL and a description of the pages you want | One result per matching page | As a job |
| [Map](https://docs.spidra.io/map.md) | A website | Its page URLs | Straight away |
| [Search](https://docs.spidra.io/search.md) | A query | Results from the web, news, images, videos, research or GitHub | As a job, or straight away with `wait` |
| [Parse](https://docs.spidra.io/parse.md) | A document by upload or link | Markdown page by page, or JSON fields | Straight away if it finishes within 20 seconds, otherwise as a job |
| [Templates](https://docs.spidra.io/templates.md) | A template and its inputs | Rows | As a run, or straight away with `wait` |

[Monitors](https://docs.spidra.io/monitors.md) run a template, a preset or a list of pages on a schedule and tell you when something changed.

## Jobs and polling

Most work takes from a few seconds to a few minutes, so it runs as a job:

1. You send the request, for example `POST /scrape`. The answer is `202` with a `jobId` (a `batchId` for batch scrapes, a `runId` for templates).
2. You ask for the job, for example `GET /scrape/{jobId}`, every few seconds. Asking is free and isn't rate limited.
3. You stop when `status` is `completed`, `failed` or `cancelled`. Until then the status says the job is waiting or working (`queued`, `pending`, `active` or `running`, depending on the tool), and scrapes, crawls and parse jobs add a `progress` message.

```json theme={null}
{ "status": "active", "progress": { "message": "Processing content with AI...", "progress": 0.5 }, "result": null, "error": null }
```

Some requests can wait for the result instead:

* **Map** always answers in the same request.
* **Search** with `"wait": true` answers in the same request if it finishes within 20 seconds.
* **Batch scrape** with `"wait": true` and up to 10 URLs answers in the same request if it finishes within 45 seconds.
* **Parse** waits up to 20 seconds by default.
* **Template runs** with `"wait": true` wait like the tool behind them.

If it doesn't finish in time you get the usual `202` and a job ID, so code that waits must also handle polling. The [Node.js and Python SDKs](https://docs.spidra.io/sdks/overview.md) do all of this for you: `scrape()`, `crawl()`, `search()` and the rest return the finished result.

## What comes back

A finished scrape always has the same outer shape:

```json theme={null}
{
  "status": "completed",
  "result": {
    "content": "...",
    "data": [
      { "url": "https://example.com", "title": "Example Domain", "markdownContent": "...", "success": true }
    ],
    "stats": { "durationMs": 5356, "inputTokens": 0, "outputTokens": 0, "totalTokens": 0 }
  }
}
```

* `result.content` is the answer.
* `result.data` has one entry per URL with that page's Markdown.
* `result.stats` says how long it took and how much AI it used.

What's in `content` depends on what you ask for:

| You send | `content` is | AI runs |
| - | - | - |
| Just the URL | The page as Markdown | No |
| A `prompt` | What you described, in a shape the AI chooses | Yes |
| A `prompt` and a `schema` | An object with exactly your fields and types, `null` where the page doesn't have something | Yes |
| A link from a [supported site](https://docs.spidra.io/sites/overview.md) with `"output": "json"` | That site's fields, such as price, rating and seller | No |

Use a [schema](https://docs.spidra.io/options/structured-output.md) whenever code reads the result. Without one, field names and types can change between pages and runs.

Batch scrapes, crawls, searches, parse jobs and template runs have their own result shapes, shown on each tool's page.

## How pages are read

By default Spidra opens each page in a real browser, so pages built with JavaScript work. You can change how a page is read:

* [Browser actions](https://docs.spidra.io/options/browser-actions.md) click, type, scroll and loop over items before the page is read.
* `"scrapeMode": "fast"` reads the page over HTTP without a browser. It's quicker for static pages; Spidra still uses a browser when a page needs one.
* [Stealth Mode](https://docs.spidra.io/options/proxies-and-captchas.md) (`"useProxy": true`) sends the request through residential proxies in the country you choose. CAPTCHAs are solved automatically either way.
* [Cookies](https://docs.spidra.io/options/logged-in-pages.md) let you read pages behind your own login.
* Links from [supported sites](https://docs.spidra.io/sites/overview.md) are read directly and come back with the site's own fields.

## Credits

You pay in credits. The main rules:

* 1 credit per page scraped, crawled or parsed.
* About 1 more credit per page when AI extracts data (it depends on the page's length).
* 1 credit per map, and 1 credit per 10 search results.
* 5 credits per CAPTCHA solved.
* Stealth Mode uses proxy bandwidth instead of credits.

**Failed work is free.** A page that can't be read, comes back blank, needs a login or answers with an HTTP error isn't charged. Neither is a map that finds nothing, a search with no results, or a crawl's navigation pages (pages read only to find the ones you asked for).

When you submit a job, Spidra holds the most it could cost and gives back the difference when it finishes. If you don't have enough credits for the hold, the request is refused with `403` before anything runs. [How credits work](https://docs.spidra.io/billing/how-credits-work.md) has the full rules and examples.

## History

Every scrape, batch, crawl, map, search and parse is saved with its inputs, result and cost. Open them in the dashboard under [Logs](https://docs.spidra.io/dashboard/logs.md), or through the API with the [log endpoints](https://docs.spidra.io/api-reference/scraping/scrape-logs.md).

## Limits

| | Limit |
| - | - |
| URLs per scrape | 3 |
| URLs per batch | 50 |
| Results per search | 20 per source |
| URLs per map | 5,000 |
| Pages per crawl | 100 on Free, 1,000 on Starter, 5,000 on Builder, 10,000 on Pro |
| Pages per parsed document | 2,000 per request |
| Upload size for parse | 50 MB, or 500 MB with a prepared upload |

Every plan has every feature. Plans differ in credits, proxy bandwidth, pages per crawl and support. Request rates are on the [Rate limits](https://docs.spidra.io/api-reference/rate-limits.md) page.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.