actions array to a URL. This one logs in to a demo site, waits for the logged-in page, then extracts from it:
actionResults:
How actions run
- In order. Spidra opens the page, runs the actions one after the other, then reads the page as it is after the last one.
- A step that fails fails the job. If an element isn’t there or can’t be used, the job stops with an error that names the step, for example
Action 1 (click "#accept-cookies") failed: no element matches "#accept-cookies" (waited 2 seconds).In a request with several URLs, the whole job fails. Failed jobs are free. optional: trueskips a step instead. Use it for things that are only sometimes there, such as a cookie banner. A skipped step is reported asskippedand added to the result’swarnings, and the next step runs.- Nothing is retried. A second try could submit a form or add an item to a cart twice, so a failed step is reported, not repeated.
- Always in a browser. A URL with actions runs in a browser, also with
scrapeMode: "fast". Site scrapers step aside for URLs with actions.
POST /scrape, which takes up to 3 URLs. Batch scraping takes plain URLs and doesn’t run actions.
Pointing at an element
Actions that work on an element take aselector. It can be any of these:
Spidra treats a target as a selector when it starts with
//, contains any of # . [ : > + ~ *, or is only HTML tag names (such as "form input"). Anything else is a description. So "button" is a selector and "the submit button" is a description.
Actions
Every action also takes these:
Action types are not case-sensitive, and a few other names work too:
fill for type, waitForSelector for waitFor, keypress for press, infiniteScroll for scrollUntilEnd, executeJavascript for evaluate. Older field names also still work: value as the element for click, check, uncheck and hover, text for the text to type, value or timeout as the milliseconds of a wait, and value as the position of a scroll. The names on this page are the ones to use.
click
waitFor.
type
value is the text to type, up to 5,000 characters. It replaces what is already in the field. To submit, click the submit button or press Enter next.
check and uncheck
ok.
hover
select
value matches an option’s visible label or its value attribute. Pass a list, such as ["Red", "Blue"], to pick several options in a multi-select. It also works on custom dropdowns that aren’t a <select>: Spidra opens them and clicks the option with that text.
press
selector, the key is pressed in that element. Without one, it goes to whatever has focus, which is the field you just typed into. Key names follow the usual names: Enter, Tab, Escape, Backspace, ArrowDown, PageDown, a single character, or a combination such as Control+A.
wait
waitFor is faster and more reliable.
waitFor
state and moves on as soon as it does. state is visible (the default), attached (in the page, even if hidden), hidden or detached (gone). If it doesn’t happen within timeout (10 seconds by default, 30 at most), the step fails.
This one clicks Remove and waits until the checkbox is gone:
waitFor counts its full timeout toward the 60 seconds of waiting a URL can have, even when it finishes sooner, so seven waitFor steps with the default timeout go over it. Set a shorter timeout when you use several.
scroll
to is a percentage of the page height (80 or "80%"), "top", "bottom", or an element (selector or description) to bring into view. pixels scrolls by that much from where the page is, and a negative number scrolls up. After a scroll, Spidra waits briefly for content it triggers to load.
scrollUntilEnd
For infinite scroll and “Load more” pages:clickSelector while that button is visible, then waits pause milliseconds for new content. It stops when two rounds in a row add nothing, or after maxScrolls rounds.
Its report says how many rounds it took and why it stopped (
end or maxScrolls):
evaluate
return. The value comes back in the step’s result, cut at 10,000 characters, and the script can be up to 10,000 characters. You can use await in it. Use evaluate to read a value the page holds, to remove an overlay, or to change the page before Spidra reads it.
screenshot
screenshotUrl. fullPage: true captures the whole scrollable page instead of the visible part. This is separate from the request’s screenshot option, which captures the page at the end.
The action report
Each entry inresult.data has actionResults when that URL had actions: one entry per action, in order.
When something needs your attention, the job’s result also has a
warnings array of plain sentences. A skipped step shows up like this:
data[i].warnings. With several URLs, each warning in the job’s warnings starts with the URL it is about.
Errors
Mistakes in the actions are caught before anything runs. The request gets a422 with one line per problem, naming the URL and the action:
422 says:
URL 1, action 2: unknown type "teleport". Use one of: click, type, check, uncheck, hover, select, press, wait, waitFor, scroll, scrollUntilEnd, evaluate, screenshot, forEach.URL 1, action 1 (wait): "duration" must be between 1 and 30000 milliseconds.URL 1: the actions wait 65 seconds in total; the limit is 60 (waits, waitFor timeouts and scrollUntilEnd pauses).URL 1: use either "instruction" (one plain-English task) or "actions" (steps), not both.
error names the step and says why:
forEach: go through a list of items
forEach finds the items in a list, such as product cards, links or rows, and reads each one. It can open each item’s own page, click each item to open it, follow “Next” pages, and run a prompt on every item.
prompt, the items come back in content (and data[0].markdownContent), numbered and separated by ---:
data[0].forEachItems says how many items were delivered.
Rules
- One forEach, and it comes last. Actions before it run once, on the page, and the forEach works on the page as they leave it. Nothing can come after it. A second forEach, or one that isn’t last, gets a
422. - Up to 50 items, across all results pages.
- Each item is cleaned (site navigation, footers and boilerplate removed) and cut at 8,000 characters.
- If it finds no items, you get the whole page instead, and the
warningssay so:forEach found no items matching "each product card with a price" (none on the page), so the whole page was returned instead. - A failed item doesn’t fail the job. It shows
[Error: ...]in its place, and thewarningssay how many failed. The job fails only when every item fails. - It stops at the page’s time limit. A URL has 4 minutes in total. If forEach is still going near the end, it stops and returns the items it has, with a warning such as
forEach stopped after 37 item(s): it reached the page's time limit. Lower maxItems or waitAfterClick to get them all.
Modes
In
click mode, if a click follows a link to another page, Spidra reads that page and goes back before the next item. For links, navigate is quicker.
Fields
Finding the items
Ininline mode with a CSS captureSelector, Spidra reads every element that matches it, so ".quote" gives you each quote block. The AI isn’t used to find the items, and observe is just a label.
In every other case, the AI finds the items from observe. It is best at finding things you can click, such as links and buttons. If it misses items that are plain text, switch to inline with a CSS captureSelector.
Without captureSelector:
inlinereads each item’s own text.navigatereads the main content of each item’s page.clickreads a dialog that the click opened, or else the page.
itemPrompt and the top-level prompt
itemPrompt runs on each item as it is read, so the AI looks at one item at a time and each item comes back already extracted. The top-level prompt runs once at the end, over all the items together. You can use both: itemPrompt to pull fields out of each page, and prompt to sort, filter or reshape the whole list.
Pagination
nextSelector and goes on. It stops at maxItems, after maxPages pages (counting the first), or when there is no next page. This example reads the 10 quotes on page 1 and 5 from page 2.
Steps on each item
actions inside a forEach run on each item after it is clicked or opened, before it is read. Here each book page is scrolled before it is read:
Click mode
This page loads a table of films when you click a year, without leaving the page. Click mode clicks two years and reads the table each one loads:Do you need forEach?
Often not. If the whole list is on one page, scrape the URL with a top-levelprompt and let the AI pull out the list. Use forEach when you need items from several results pages, the details are on each item’s own page or behind a click, or you want each item extracted on its own with itemPrompt.
More forEach patterns are in Browser actions examples.
instruction: one task in plain English
Instead of steps, a URL can have aninstruction, a single task in plain English that the AI carries out before the page is read:
instruction always runs in a browser, also with scrapeMode: "fast". If the AI can’t carry it out, the job fails. A URL takes either instruction or actions, not both. Use actions when you know the steps: they are faster, cheaper and do the same thing every time.
Limits
Credits
Actions don’t change the price of a scrape: a URL costs 1 credit, however many steps it runs. A few things add to that:- forEach in
navigatemode adds 1 credit for each item page it opens. Items that fail are free. - forEach pagination adds 1 credit for each further results page.
- The AI used to find a plain-English target, to find forEach items, and to run
itemPromptis billed by tokens, like aprompt. Its tokens are added to the job’s own AI tokens and rounded up once for the whole job. CSS selectors and XPath use no AI.
itemPrompt, and the warnings say so. A job that fails is free. See How credits work.
Browser actions examples
Copy-paste requests that run against public test sites
Submit a scrape job
Every field of POST /scrape