> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Browser actions

> Click, type, wait, scroll, run JavaScript and loop over items in a real browser before Spidra reads the page.

Some pages only show what you want after someone does something: logs in, searches, picks an option, scrolls, or clicks "Load more". Browser actions let Spidra do those steps in a real browser before it reads the page.

Add an `actions` array to a URL. This one logs in to a demo site, waits for the logged-in page, then extracts from it:

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com/login",
    "actions": [
      { "type": "type",    "selector": "#username", "value": "demo" },
      { "type": "type",    "selector": "#password", "value": "demo" },
      { "type": "click",   "selector": "input[type='submit']" },
      { "type": "waitFor", "selector": "a[href='/logout']" }
    ]
  }],
  "prompt": "List the first 3 quotes with their authors",
  "output": "json"
}
```

Each URL's result says how every step went, in `actionResults`:

```json theme={null}
"actionResults": [
  { "index": 1, "type": "type",    "status": "ok", "durationMs": 16 },
  { "index": 2, "type": "type",    "status": "ok", "durationMs": 93 },
  { "index": 3, "type": "click",   "status": "ok", "durationMs": 778 },
  { "index": 4, "type": "waitFor", "status": "ok", "durationMs": 5 }
]
```

## How actions run

* **In order.** Spidra opens the page, runs the actions one after the other, then reads the page as it is after the last one.
* **A step that fails fails the job.** If an element isn't there or can't be used, the job stops with an error that names the step, for example `Action 1 (click "#accept-cookies") failed: no element matches "#accept-cookies" (waited 2 seconds).` In a request with several URLs, the whole job fails. Failed jobs are free.
* **`optional: true` skips a step instead.** Use it for things that are only sometimes there, such as a cookie banner. A skipped step is reported as `skipped` and added to the result's `warnings`, and the next step runs.
* **Nothing is retried.** A second try could submit a form or add an item to a cart twice, so a failed step is reported, not repeated.
* **Always in a browser.** A URL with actions runs in a browser, also with `scrapeMode: "fast"`. [Site scrapers](https://docs.spidra.io/sites/amazon.md) step aside for URLs with actions.

Actions are set per URL on [`POST /scrape`](https://docs.spidra.io/api-reference/scraping/scrape), which takes up to 3 URLs. [Batch scraping](https://docs.spidra.io/batch-scrape.md) takes plain URLs and doesn't run actions.

## Pointing at an element

Actions that work on an element take a `selector`. It can be any of these:

| Kind | Example | How Spidra finds it |
| - | - | - |
| CSS selector | `"#username"`, `"li.next > a"`, `"input[type='submit']"` | Directly |
| XPath | `"//button[contains(., 'Accept')]"` | Directly |
| Plain English | `"the Next page link at the bottom"` | The AI finds the element on the page, then Spidra uses it like a selector |

Spidra treats a target as a selector when it starts with `//`, contains any of `# . [ : > + ~ *`, or is only HTML tag names (such as `"form input"`). Anything else is a description. So `"button"` is a selector and `"the submit button"` is a description.

```json theme={null}
{ "type": "click", "selector": "li.next > a" }
{ "type": "click", "selector": "the Next page link at the bottom" }
```

Plain English is handy when a page has no stable selector, but a CSS selector is faster and costs no AI. Use CSS for steps you run often. Either way, the text you type is never sent to the AI, and it is masked in your logs.

## Actions

| Action | Fields | What it does |
| - | - | - |
| `click` | `selector` | Clicks a button, link, tab or toggle |
| `type` | `selector`, `value` | Fills a text field with `value`. Without `selector` it fills the first visible text field |
| `check`, `uncheck` | `selector` | Ticks or clears a checkbox |
| `hover` | `selector` | Moves the mouse over an element, for menus and captions that open on hover |
| `select` | `selector`, `value` | Picks a dropdown option by its label or its value. `value` can be a list for a multi-select |
| `press` | `key`, `selector` (optional) | Presses a key, such as `Enter`, `Tab`, `Escape` or `Control+A`, in an element or on the page |
| `wait` | `duration` | Pauses for a number of milliseconds, up to 30,000 |
| `waitFor` | `selector`, `state`, `timeout` | Waits until an element is visible, or `attached`, `hidden` or `detached` |
| `scroll` | `to` or `pixels` | Scrolls to a percentage, the top, the bottom or an element, or by a number of pixels |
| `scrollUntilEnd` | `maxScrolls`, `pause`, `clickSelector` | Keeps scrolling, or clicking a "Load more" button, until the page stops growing |
| `evaluate` | `script` | Runs JavaScript in the page and returns the value |
| `screenshot` | `fullPage` | Takes a screenshot at this point and returns a link |
| `forEach` | `observe`, `mode` and [more](#foreach-go-through-a-list-of-items) | Goes through a list of items and reads each one. It must be the last action |

Every action also takes these:

| Field | Applies to | Default | What it does |
| - | - | - | - |
| `optional` | Every action | `false` | If the step fails, skip it and carry on instead of failing the job |
| `timeout` | Actions with a `selector` | `10000` | How long, in milliseconds, to look for the element (100 to 30,000). For `waitFor`, how long to wait |

Action types are not case-sensitive, and a few other names work too: `fill` for `type`, `waitForSelector` for `waitFor`, `keypress` for `press`, `infiniteScroll` for `scrollUntilEnd`, `executeJavascript` for `evaluate`. Older field names also still work: `value` as the element for `click`, `check`, `uncheck` and `hover`, `text` for the text to type, `value` or `timeout` as the milliseconds of a `wait`, and `value` as the position of a `scroll`. The names on this page are the ones to use.

### click

```json theme={null}
{ "type": "click", "selector": "li.next > a" }
```

If a click loads a new page, Spidra waits for it to load before the next step. For content that a click loads into the same page, follow it with `waitFor`.

### type

```json theme={null}
{ "type": "type", "selector": "#username", "value": "demo" }
```

`value` is the text to type, up to 5,000 characters. It replaces what is already in the field. To submit, click the submit button or press `Enter` next.

### check and uncheck

```json theme={null}
{
  "urls": [{
    "url": "https://the-internet.herokuapp.com/checkboxes",
    "actions": [
      { "type": "check",   "selector": "#checkboxes input:nth-of-type(1)" },
      { "type": "uncheck", "selector": "#checkboxes input:nth-of-type(2)" }
    ]
  }]
}
```

Ticking a box that is already ticked, or clearing one that is already clear, does nothing and still counts as `ok`.

### hover

```json theme={null}
{ "type": "hover", "selector": ".figure" }
```

Use it for dropdown menus, tooltips and captions that only appear while the mouse is over something.

### select

```json theme={null}
{ "type": "select", "selector": "#dropdown", "value": "Option 2" }
```

`value` matches an option's visible label or its `value` attribute. Pass a list, such as `["Red", "Blue"]`, to pick several options in a multi-select. It also works on custom dropdowns that aren't a `<select>`: Spidra opens them and clicks the option with that text.

### press

```json theme={null}
{ "type": "press", "key": "Enter" }
{ "type": "press", "selector": "#q", "key": "Enter" }
```

With a `selector`, the key is pressed in that element. Without one, it goes to whatever has focus, which is the field you just typed into. Key names follow the usual names: `Enter`, `Tab`, `Escape`, `Backspace`, `ArrowDown`, `PageDown`, a single character, or a combination such as `Control+A`.

### wait

```json theme={null}
{ "type": "wait", "duration": 2000 }
```

A fixed pause, in milliseconds, up to 30,000. When you know what you're waiting for, `waitFor` is faster and more reliable.

### waitFor

```json theme={null}
{ "type": "waitFor", "selector": "#finish", "timeout": 15000 }
```

Waits until the element reaches `state` and moves on as soon as it does. `state` is `visible` (the default), `attached` (in the page, even if hidden), `hidden` or `detached` (gone). If it doesn't happen within `timeout` (10 seconds by default, 30 at most), the step fails.

This one clicks **Remove** and waits until the checkbox is gone:

```json theme={null}
{
  "urls": [{
    "url": "https://the-internet.herokuapp.com/dynamic_controls",
    "actions": [
      { "type": "click",   "selector": "#checkbox-example button" },
      { "type": "waitFor", "selector": "#checkbox", "state": "detached" }
    ]
  }]
}
```

A `waitFor` counts its full `timeout` toward the [60 seconds of waiting](#limits) a URL can have, even when it finishes sooner, so seven `waitFor` steps with the default timeout go over it. Set a shorter `timeout` when you use several.

### scroll

```json theme={null}
{ "type": "scroll", "to": 80 }
{ "type": "scroll", "to": "bottom" }
{ "type": "scroll", "to": "li.next" }
{ "type": "scroll", "pixels": 600 }
```

`to` is a percentage of the page height (`80` or `"80%"`), `"top"`, `"bottom"`, or an element (selector or description) to bring into view. `pixels` scrolls by that much from where the page is, and a negative number scrolls up. After a scroll, Spidra waits briefly for content it triggers to load.

### scrollUntilEnd

For infinite scroll and "Load more" pages:

```json theme={null}
{ "type": "scrollUntilEnd", "maxScrolls": 15, "pause": 1000 }
{ "type": "scrollUntilEnd", "clickSelector": "the Load more button", "maxScrolls": 10 }
```

Each round scrolls to the bottom, or clicks `clickSelector` while that button is visible, then waits `pause` milliseconds for new content. It stops when two rounds in a row add nothing, or after `maxScrolls` rounds.

| Field | Default | Range |
| - | - | - |
| `maxScrolls` | `10` | 1 to 30 |
| `pause` | `1500` | 200 to 5,000 ms |
| `clickSelector` | none | A selector or description of a "Load more" button |

Its report says how many rounds it took and why it stopped (`end` or `maxScrolls`):

```json theme={null}
{ "index": 1, "type": "scrollUntilEnd", "status": "ok", "durationMs": 11104, "result": { "scrolls": 11, "stoppedBecause": "end" } }
```

### evaluate

```json theme={null}
{ "type": "evaluate", "script": "document.querySelectorAll('.quote').length" }
{ "type": "evaluate", "script": "const tags = [...document.querySelectorAll('.tag-item a')].map(a => a.innerText); return { count: tags.length, tags };" }
```

Runs JavaScript in the page. A single expression is returned as is. A script with statements needs `return`. The value comes back in the step's `result`, cut at 10,000 characters, and the script can be up to 10,000 characters. You can use `await` in it. Use `evaluate` to read a value the page holds, to remove an overlay, or to change the page before Spidra reads it.

```json theme={null}
{ "index": 1, "type": "evaluate", "status": "ok", "durationMs": 2, "result": { "count": 10, "tags": ["love", "inspirational", "life", "humor", "books", "reading", "friendship", "friends", "truth", "simile"] } }
```

### screenshot

```json theme={null}
{ "type": "screenshot", "fullPage": true }
```

Takes a screenshot at that point in the steps, which helps you see what the page looked like before a step that fails. The link is in the step's `screenshotUrl`. `fullPage: true` captures the whole scrollable page instead of the visible part. This is separate from the request's `screenshot` option, which captures the page at the end.

## The action report

Each entry in `result.data` has `actionResults` when that URL had actions: one entry per action, in order.

| Field | What it is |
| - | - |
| `index` | The step's position, starting at 1 |
| `type` | The action type, as Spidra ran it (`waitFor` for `waitForSelector`, for example) |
| `status` | `ok`, `skipped` (an optional step that couldn't be done) or `failed` |
| `durationMs` | How long the step took |
| `error` | Why the step was skipped or failed |
| `result` | What `evaluate` returned, or how `scrollUntilEnd` went |
| `screenshotUrl` | The link from a `screenshot` step |

When something needs your attention, the job's result also has a `warnings` array of plain sentences. A skipped step shows up like this:

```json theme={null}
{
  "warnings": [
    "Action 1 (click) was skipped: no element matches \"#accept-cookies\" (waited 2 seconds)."
  ],
  "data": [{
    "url": "https://quotes.toscrape.com",
    "actionResults": [
      { "index": 1, "type": "click", "status": "skipped", "durationMs": 2006, "error": "no element matches \"#accept-cookies\" (waited 2 seconds)." },
      { "index": 2, "type": "click", "status": "ok", "durationMs": 308 }
    ]
  }]
}
```

That came from a page with no cookie banner, where the first step is optional:

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [
      { "type": "click", "selector": "#accept-cookies", "optional": true, "timeout": 2000 },
      { "type": "click", "selector": "li.next > a" }
    ]
  }]
}
```

The same warnings also appear on that URL's own `data[i].warnings`. With several URLs, each warning in the job's `warnings` starts with the URL it is about.

## Errors

Mistakes in the actions are caught before anything runs. The request gets a `422` with one line per problem, naming the URL and the action:

```json theme={null}
{
  "status": "error",
  "message": "Request validation failed. Fix the errors below and try again.",
  "errors": [
    "URL 1, action 1: forEach must be the last action (it collects the results, so nothing can run after it)."
  ]
}
```

Other examples of what a `422` says:

* `URL 1, action 2: unknown type "teleport". Use one of: click, type, check, uncheck, hover, select, press, wait, waitFor, scroll, scrollUntilEnd, evaluate, screenshot, forEach.`
* `URL 1, action 1 (wait): "duration" must be between 1 and 30000 milliseconds.`
* `URL 1: the actions wait 65 seconds in total; the limit is 60 (waits, waitFor timeouts and scrollUntilEnd pauses).`
* `URL 1: use either "instruction" (one plain-English task) or "actions" (steps), not both.`

A step that fails while the job runs fails the job, which is free. The job's `error` names the step and says why:

```json theme={null}
{
  "status": "failed",
  "error": "Failed to scrape https://quotes.toscrape.com: Action 1 (click \"#accept-cookies\") failed: no element matches \"#accept-cookies\" (waited 2 seconds). Add \"optional\": true to that action to skip it when it can't be done."
}
```

| The reason says | What it means |
| - | - |
| `no element matches "..." (waited N seconds)` | Nothing on the page matches the selector. Check it in your browser's developer tools, or wait for it first with `waitFor` |
| `"..." was found but couldn't be used: it stayed hidden, disabled or covered` | The element is there, but something such as a banner or a hidden menu is in the way |
| `couldn't find "..." on the page` | The AI found nothing matching a plain-English description |
| `the option [...] isn't in that dropdown` | No option in the dropdown has that label or value |
| `"..." isn't a key name` | `press` got a key name it doesn't know |
| `the script threw an error: ...` or `the script has a syntax error: ...` | The `evaluate` script failed |
| `it didn't succeed within N seconds` | The step ran out of time |

## forEach: go through a list of items

`forEach` finds the items in a list, such as product cards, links or rows, and reads each one. It can open each item's own page, click each item to open it, follow "Next" pages, and run a prompt on every item.

```json theme={null}
{
  "urls": [{
    "url": "https://books.toscrape.com/catalogue/category/books/travel_2/index.html",
    "actions": [{
      "type": "forEach",
      "observe": "each book title link in the product grid",
      "mode": "navigate",
      "captureSelector": "article.product_page",
      "maxItems": 3,
      "waitAfterClick": 500,
      "itemPrompt": "Return JSON: {title, price, availability}"
    }]
  }]
}
```

With no top-level `prompt`, the items come back in `content` (and `data[0].markdownContent`), numbered and separated by `---`:

```
## Item 1

{
  "title": "It's Only the Himalayas",
  "price": "£45.17",
  "availability": "In stock (19 available)"
}

---

## Item 2

{
  "title": "Full Moon over Noah’s Ark: An Odyssey to Mount Ararat and Beyond",
  "price": "£49.43",
  "availability": "In stock (15 available)"
}

---

## Item 3

{
  "title": "See America: A Celebration of Our National Parks & Treasured Sites",
  "price": "£48.87",
  "availability": "In stock (14 available)"
}
```

`data[0].forEachItems` says how many items were delivered.

### Rules

* **One forEach, and it comes last.** Actions before it run once, on the page, and the forEach works on the page as they leave it. Nothing can come after it. A second forEach, or one that isn't last, gets a `422`.
* **Up to 50 items**, across all results pages.
* **Each item is cleaned** (site navigation, footers and boilerplate removed) and cut at 8,000 characters.
* **If it finds no items**, you get the whole page instead, and the `warnings` say so: `forEach found no items matching "each product card with a price" (none on the page), so the whole page was returned instead.`
* **A failed item doesn't fail the job.** It shows `[Error: ...]` in its place, and the `warnings` say how many failed. The job fails only when every item fails.
* **It stops at the page's time limit.** A URL has 4 minutes in total. If forEach is still going near the end, it stops and returns the items it has, with a warning such as `forEach stopped after 37 item(s): it reached the page's time limit. Lower maxItems or waitAfterClick to get them all.`

### Modes

| Mode | What it does with each item | Use it for |
| - | - | - |
| `inline` | Reads the item where it is, without clicking | Cards, rows and list items that already show what you want |
| `navigate` | Opens the page the item links to, reads it, and comes back | Listings where the details are on each item's own page |
| `click` (default) | Clicks the item, reads what opens (a dialog, a panel, a loaded table), then closes it | Accordions, modals and tabs that load content in place |

In `click` mode, if a click follows a link to another page, Spidra reads that page and goes back before the next item. For links, `navigate` is quicker.

### Fields

| Field | Required | Default | What it does |
| - | - | - | - |
| `observe` | Yes | | The items to go through, in plain English, such as `"each book title link in the product grid"`. Describe what the items are, not what to do with them |
| `mode` | No | `click` | `inline`, `navigate` or `click` |
| `captureSelector` | No | | The part of the page to read for each item, as a selector or a description. If it isn't found for an item, that item fails |
| `maxItems` | No | `50` | The most items to process, 1 to 50, across all results pages |
| `waitAfterClick` | No | `2500` | Milliseconds to wait after clicking or opening an item before reading it, 0 to 10,000 |
| `itemPrompt` | No | | A prompt the AI runs on each item, up to 2,000 characters |
| `actions` | No | | Up to 10 steps to run on each item after it is clicked or opened (`click` and `navigate` modes). The same actions as above, without forEach. A failed step fails that item |
| `pagination.nextSelector` | With `pagination` | | The next-page link or button, as a selector or a description |
| `pagination.maxPages` | No | `5` | The most results pages to go through, counting the first, 1 to 10 |

### Finding the items

In `inline` mode with a CSS `captureSelector`, Spidra reads every element that matches it, so `".quote"` gives you each quote block. The AI isn't used to find the items, and `observe` is just a label.

In every other case, the AI finds the items from `observe`. It is best at finding things you can click, such as links and buttons. If it misses items that are plain text, switch to `inline` with a CSS `captureSelector`.

Without `captureSelector`:

* `inline` reads each item's own text.
* `navigate` reads the main content of each item's page.
* `click` reads a dialog that the click opened, or else the page.

### itemPrompt and the top-level prompt

`itemPrompt` runs on each item as it is read, so the AI looks at one item at a time and each item comes back already extracted. The top-level `prompt` runs once at the end, over all the items together. You can use both: `itemPrompt` to pull fields out of each page, and `prompt` to sort, filter or reshape the whole list.

### Pagination

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [{
      "type": "forEach",
      "observe": "each quote block",
      "mode": "inline",
      "captureSelector": ".quote",
      "maxItems": 15,
      "pagination": { "nextSelector": "li.next > a", "maxPages": 2 }
    }]
  }]
}
```

After the items on a page, Spidra clicks `nextSelector` and goes on. It stops at `maxItems`, after `maxPages` pages (counting the first), or when there is no next page. This example reads the 10 quotes on page 1 and 5 from page 2.

### Steps on each item

`actions` inside a forEach run on each item after it is clicked or opened, before it is read. Here each book page is scrolled before it is read:

```json theme={null}
{
  "urls": [{
    "url": "https://books.toscrape.com/catalogue/category/books/poetry_23/index.html",
    "actions": [{
      "type": "forEach",
      "observe": "each book title link in the product grid",
      "mode": "navigate",
      "captureSelector": "article.product_page",
      "maxItems": 2,
      "waitAfterClick": 500,
      "actions": [
        { "type": "scroll", "to": "bottom" }
      ],
      "itemPrompt": "Return JSON: {title, price, description}"
    }]
  }]
}
```

### Click mode

This page loads a table of films when you click a year, without leaving the page. Click mode clicks two years and reads the table each one loads:

```json theme={null}
{
  "urls": [{
    "url": "https://www.scrapethissite.com/pages/ajax-javascript/",
    "actions": [{
      "type": "forEach",
      "observe": "each year link (2015, 2014, 2013 ...)",
      "mode": "click",
      "captureSelector": "table.table",
      "maxItems": 2,
      "waitAfterClick": 2000
    }]
  }]
}
```

### Do you need forEach?

Often not. If the whole list is on one page, scrape the URL with a top-level `prompt` and let the AI pull out the list. Use forEach when you need items from several results pages, the details are on each item's own page or behind a click, or you want each item extracted on its own with `itemPrompt`.

More forEach patterns are in [Browser actions examples](https://docs.spidra.io/options/browser-action-recipes.md).

## instruction: one task in plain English

Instead of steps, a URL can have an `instruction`, a single task in plain English that the AI carries out before the page is read:

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "instruction": "Click the Next page link"
  }]
}
```

An `instruction` always runs in a browser, also with `scrapeMode: "fast"`. If the AI can't carry it out, the job fails. A URL takes either `instruction` or `actions`, not both. Use `actions` when you know the steps: they are faster, cheaper and do the same thing every time.

## Limits

| Limit | Value |
| - | - |
| Actions per URL | 50 |
| A single `wait`, or any `timeout` | 30 seconds |
| Total waiting per URL (`wait` durations, `waitFor` timeouts and `scrollUntilEnd` pauses added up) | 60 seconds |
| How long an element action looks for its element | 10 seconds by default |
| Text in a `type` step | 5,000 characters |
| `evaluate` script, and the value it returns | 10,000 characters each |
| `scrollUntilEnd` rounds | 30 |
| forEach per URL | 1, as the last action |
| forEach items | 50 |
| forEach results pages | 10 |
| forEach steps per item | 10 |
| forEach item content | 8,000 characters |
| Time per URL, actions and forEach included | 4 minutes |

## Credits

Actions don't change the price of a scrape: a URL costs 1 credit, however many steps it runs. A few things add to that:

* **forEach in `navigate` mode** adds 1 credit for each item page it opens. Items that fail are free.
* **forEach pagination** adds 1 credit for each further results page.
* **The AI** used to find a plain-English target, to find forEach items, and to run `itemPrompt` is billed by tokens, like a `prompt`. Its tokens are added to the job's own AI tokens and rounded up once for the whole job. CSS selectors and XPath use no AI.

A job never charges more credits than you have. If credits run out during a forEach, it stops, or keeps items without their `itemPrompt`, and the `warnings` say so. A job that fails is free. See [How credits work](https://docs.spidra.io/billing/how-credits-work.md).

> **[Browser actions examples](https://docs.spidra.io/options/browser-action-recipes.md)**
>
> Copy-paste requests that run against public test sites

> **[Submit a scrape job](https://docs.spidra.io/api-reference/scraping/scrape.md)**
>
> Every field of POST /scrape


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.