> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Browser action recipes

> Copy-paste browser action requests that run against public test sites: log in, search, wait, pick options, infinite scroll, JavaScript, screenshots and forEach.

Every example on this page runs against a public site built for scraping practice: [quotes.toscrape.com](https://quotes.toscrape.com), [books.toscrape.com](https://books.toscrape.com), [the-internet.herokuapp.com](https://the-internet.herokuapp.com) and [scrapethissite.com](https://www.scrapethissite.com). Send any of them to [`POST /scrape`](https://docs.spidra.io/api-reference/scraping/scrape) as they are, then poll `GET /scrape/{jobId}` for the result.

For what each action does, see [Browser actions](https://docs.spidra.io/options/browser-actions.md).

## 1. No actions needed

When everything you want is on one page, you don't need actions. Send the URL with a prompt.

```json theme={null}
{
  "urls": [{ "url": "https://quotes.toscrape.com" }],
  "prompt": "List every quote and its author. Return a JSON array: [{quote, author}]",
  "output": "json"
}
```

```json theme={null}
{
  "content": [
    { "quote": "The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.", "author": "Albert Einstein" },
    { "quote": "It is our choices, Harry, that show what we truly are, far more than our abilities.", "author": "J.K. Rowling" },
    ...
  ]
}
```

## 2. Log in, then scrape

Fill in a login form, submit it, and wait for the logged-in page before Spidra reads it.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com/login",
    "actions": [
      { "type": "type",    "selector": "#username", "value": "demo" },
      { "type": "type",    "selector": "#password", "value": "demo" },
      { "type": "click",   "selector": "input[type='submit']" },
      { "type": "waitFor", "selector": "a[href='/logout']" }
    ]
  }],
  "prompt": "List the first 3 quotes with their authors",
  "output": "json"
}
```

The `waitFor` on the Logout link makes sure the login worked. If it didn't, the job fails at step 4 and costs nothing. What you type is never sent to the AI and is masked in your logs. If you already have a session, [cookies](https://docs.spidra.io/options/logged-in-pages.md) skip the form.

## 3. Click away a banner only if it's there

Mark a step `optional` when it only applies sometimes, such as a cookie banner. If it can't be done, it is skipped and the job goes on.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [
      { "type": "click", "selector": "#accept-cookies", "optional": true, "timeout": 2000 },
      { "type": "click", "selector": "li.next > a" }
    ]
  }]
}
```

This page has no banner, so the result says so:

```json theme={null}
{
  "warnings": ["Action 1 (click) was skipped: no element matches \"#accept-cookies\" (waited 2 seconds)."],
  "data": [{
    "actionResults": [
      { "index": 1, "type": "click", "status": "skipped", "durationMs": 2006, "error": "no element matches \"#accept-cookies\" (waited 2 seconds)." },
      { "index": 2, "type": "click", "status": "ok", "durationMs": 308 }
    ]
  }]
}
```

A short `timeout` keeps an optional step from waiting the full 10 seconds.

## 4. Search and press Enter

Type into a search box, press `Enter`, and wait for the results.

```json theme={null}
{
  "urls": [{
    "url": "https://www.scrapethissite.com/pages/forms/",
    "actions": [
      { "type": "type",    "selector": "#q", "value": "Boston" },
      { "type": "press",   "key": "Enter" },
      { "type": "waitFor", "selector": "tr.team" }
    ]
  }],
  "prompt": "List each team name, year and wins",
  "output": "json"
}
```

```json theme={null}
{
  "content": [
    { "team_name": "Boston Bruins", "year": 1990, "wins": 44 },
    { "team_name": "Boston Bruins", "year": 1991, "wins": 36 },
    { "team_name": "Boston Bruins", "year": 1992, "wins": 51 },
    ...
  ]
}
```

## 5. Wait for content that loads later

The button on this page starts a loader, and the text appears a few seconds later. `waitFor` moves on as soon as it shows up.

```json theme={null}
{
  "urls": [{
    "url": "https://the-internet.herokuapp.com/dynamic_loading/2",
    "actions": [
      { "type": "click",   "selector": "#start button" },
      { "type": "waitFor", "selector": "#finish", "timeout": 15000 }
    ]
  }]
}
```

The page content now includes `Hello World!`. Without the `waitFor`, Spidra would read the page before it loads.

## 6. Click with a plain-English description

When an element has no stable selector, describe it. The AI finds it, then Spidra clicks it.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [
      { "type": "click", "selector": "the Next page link at the bottom" }
    ]
  }]
}
```

This returns page 2. Finding the element uses a little AI, billed by tokens.

## 7. Pick a dropdown option and read it back

`select` picks an option by its label or value. `evaluate` runs JavaScript and returns what it gives back, here the dropdown's value.

```json theme={null}
{
  "urls": [{
    "url": "https://the-internet.herokuapp.com/dropdown",
    "actions": [
      { "type": "select",   "selector": "#dropdown", "value": "Option 2" },
      { "type": "evaluate", "script": "document.querySelector('#dropdown').value" }
    ]
  }]
}
```

```json theme={null}
"actionResults": [
  { "index": 1, "type": "select",   "status": "ok", "durationMs": 5 },
  { "index": 2, "type": "evaluate", "status": "ok", "durationMs": 1, "result": "2" }
]
```

## 8. Hover to reveal hidden text

```json theme={null}
{
  "urls": [{
    "url": "https://the-internet.herokuapp.com/hovers",
    "actions": [
      { "type": "hover",    "selector": ".figure" },
      { "type": "evaluate", "script": "document.querySelector('.figcaption h5').innerText" }
    ]
  }]
}
```

The `evaluate` step returns `"name: user1"`, the caption that only shows while the mouse is over the first picture.

## 9. Load an infinite scroll page

`scrollUntilEnd` keeps scrolling until the page stops growing. This page loads 10 quotes at a time, up to 100.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com/scroll",
    "actions": [
      { "type": "scrollUntilEnd", "maxScrolls": 15, "pause": 1000 },
      { "type": "evaluate", "script": "document.querySelectorAll('.quote').length" }
    ]
  }]
}
```

```json theme={null}
"actionResults": [
  { "index": 1, "type": "scrollUntilEnd", "status": "ok", "durationMs": 11104, "result": { "scrolls": 11, "stoppedBecause": "end" } },
  { "index": 2, "type": "evaluate", "status": "ok", "durationMs": 2, "result": 100 }
]
```

On a page with a "Load more" button, add `"clickSelector": "the Load more button"` (or its CSS selector) and Spidra clicks it each round while it is there.

## 10. Return data with JavaScript

A script with statements needs `return`. The value comes back in `actionResults`, so you can collect exactly what you need without a prompt.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [
      { "type": "evaluate", "script": "const tags = [...document.querySelectorAll('.tag-item a')].map(a => a.innerText); return { count: tags.length, tags };" }
    ]
  }]
}
```

```json theme={null}
"result": { "count": 10, "tags": ["love", "inspirational", "life", "humor", "books", "reading", "friendship", "friends", "truth", "simile"] }
```

## 11. Screenshot partway through

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [
      { "type": "click", "selector": "li.next > a" },
      { "type": "screenshot", "fullPage": true }
    ]
  }]
}
```

The `screenshot` step's report has a `screenshotUrl` with an image of page 2.

## 12. forEach: read every item, across pages

`inline` mode reads each item where it is. A CSS `captureSelector` lists the items directly, and `pagination` follows the Next link.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [{
      "type": "forEach",
      "observe": "each quote block",
      "mode": "inline",
      "captureSelector": ".quote",
      "maxItems": 15,
      "pagination": { "nextSelector": "li.next > a", "maxPages": 2 }
    }]
  }]
}
```

```
## Item 1

“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”
by Albert Einstein (about)
Tags: change deep-thoughts thinking world

---

## Item 2

“It is our choices, Harry, that show what we truly are, far more than our abilities.”
by J.K. Rowling (about)
Tags: abilities choices

---

...
```

That is 10 quotes from page 1 and 5 from page 2. `maxPages` counts the first page.

## 13. forEach: open each item's page

`navigate` mode opens the page each item links to. `itemPrompt` pulls fields out of each page as it is read.

```json theme={null}
{
  "urls": [{
    "url": "https://books.toscrape.com/catalogue/category/books/travel_2/index.html",
    "actions": [{
      "type": "forEach",
      "observe": "each book title link in the product grid",
      "mode": "navigate",
      "captureSelector": "article.product_page",
      "maxItems": 3,
      "waitAfterClick": 500,
      "itemPrompt": "Return JSON: {title, price, availability}"
    }]
  }]
}
```

```
## Item 1

{
  "title": "It's Only the Himalayas",
  "price": "£45.17",
  "availability": "In stock (19 available)"
}

---

## Item 2

{
  "title": "Full Moon over Noah’s Ark: An Odyssey to Mount Ararat and Beyond",
  "price": "£49.43",
  "availability": "In stock (15 available)"
}

---

## Item 3

{
  "title": "See America: A Celebration of Our National Parks & Treasured Sites",
  "price": "£48.87",
  "availability": "In stock (14 available)"
}
```

Each page opened costs 1 credit on top of the 1 for the URL, and `itemPrompt` adds its tokens.

## 14. Click to a category first, then forEach

Actions before a forEach run once. Here a plain-English click opens the Science category, and the forEach reads its books.

```json theme={null}
{
  "urls": [{
    "url": "https://books.toscrape.com",
    "actions": [
      { "type": "click", "selector": "the Science category in the left sidebar" },
      {
        "type": "forEach",
        "observe": "each book card",
        "mode": "inline",
        "captureSelector": "article.product_pod",
        "maxItems": 5,
        "itemPrompt": "Return JSON: {title, price}"
      }
    ]
  }]
}
```

## 15. forEach: click each item and read what loads

`click` mode clicks each item and reads what it opens on the same page. On this page, clicking a year loads that year's films into a table.

```json theme={null}
{
  "urls": [{
    "url": "https://www.scrapethissite.com/pages/ajax-javascript/",
    "actions": [{
      "type": "forEach",
      "observe": "each year link (2015, 2014, 2013 ...)",
      "mode": "click",
      "captureSelector": "table.table",
      "maxItems": 2,
      "waitAfterClick": 2000
    }]
  }]
}
```

Item 1 is the 2015 table (Spotlight, Mad Max: Fury Road, The Revenant and the rest) and item 2 is 2014.

## 16. forEach: run steps on each item

`actions` inside a forEach run on each item after it is opened. Here each book page is scrolled to the bottom before it is read.

```json theme={null}
{
  "urls": [{
    "url": "https://books.toscrape.com/catalogue/category/books/poetry_23/index.html",
    "actions": [{
      "type": "forEach",
      "observe": "each book title link in the product grid",
      "mode": "navigate",
      "captureSelector": "article.product_page",
      "maxItems": 2,
      "waitAfterClick": 500,
      "actions": [
        { "type": "scroll", "to": "bottom" }
      ],
      "itemPrompt": "Return JSON: {title, price, description}"
    }]
  }]
}
```

## 17. Several categories in one request

A request takes up to 3 URLs, each with its own actions. They run in parallel and `data` has one entry per URL.

```json theme={null}
{
  "urls": [
    {
      "url": "https://books.toscrape.com/catalogue/category/books/mystery_3/index.html",
      "actions": [{ "type": "forEach", "observe": "each book card", "mode": "inline", "captureSelector": "article.product_pod", "maxItems": 3 }]
    },
    {
      "url": "https://books.toscrape.com/catalogue/category/books/travel_2/index.html",
      "actions": [{ "type": "forEach", "observe": "each book card", "mode": "inline", "captureSelector": "article.product_pod", "maxItems": 3 }]
    },
    {
      "url": "https://books.toscrape.com/catalogue/category/books/historical-fiction_4/index.html",
      "actions": [{ "type": "forEach", "observe": "each book card", "mode": "inline", "captureSelector": "article.product_pod", "maxItems": 3 }]
    }
  ],
  "prompt": "Return a JSON array of every book: {category, title, price}",
  "output": "json"
}
```

## 18. forEach plus a final prompt

`itemPrompt` works on one item at a time. A top-level `prompt` then runs once over all of them, here to sort the list.

```json theme={null}
{
  "urls": [{
    "url": "https://quotes.toscrape.com",
    "actions": [{
      "type": "forEach",
      "observe": "each quote block",
      "mode": "inline",
      "captureSelector": ".quote",
      "maxItems": 20,
      "pagination": { "nextSelector": "li.next > a", "maxPages": 2 }
    }]
  }],
  "prompt": "Return a JSON array of {author, quote}, sorted by author name",
  "output": "json"
}
```

## Choosing a pattern

| You want to | Use |
| - | - |
| Read a list that fits on one page | A top-level `prompt`, no actions (1) |
| Get past a login form | `type`, `click`, then `waitFor` (2) |
| Dismiss something that is only sometimes there | `optional: true` (3) |
| Submit a search | `type` and `press` `Enter` (4) |
| Wait for content that loads late | `waitFor` (5) |
| Load an infinite scroll or "Load more" page | `scrollUntilEnd` (9) |
| Read a value the page holds | `evaluate` (7, 10) |
| Collect items across results pages | forEach `inline` with `pagination` (12, 18) |
| Get details from each item's own page | forEach `navigate` (13, 16) |
| Read content that opens on a click | forEach `click` (15) |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.