Skip to main content
GET
Get Scrape Job Status

Polling Pattern

Scrape jobs are processed asynchronously. When you submit a job you get a jobId back immediately. You then poll this endpoint every 2-5 seconds until status is completed or failed.

Status Values

Response Structure

When status is completed, everything you need is inside result.

result.content

This is the main output field. What it contains depends on whether you provided a prompt:
  • With prompt: the AI-extracted result, formatted according to output ("markdown" or "json")
  • Without prompt: the raw scraped page content as markdown

Detecting a failed or empty extraction

Two separate flags tell you when AI extraction did not give you usable data. They mean different things: A page that simply does not contain what you asked for is not an ai_extraction_failed case. Check extraction_empty (or look for null fields yourself) before trusting the result. Each entry in result.data also has a lowContent flag. It is true when the page produced very little text, which usually means a bot check, a cookie wall or a dead redirect and not the real page.

Protected pages

If a page is behind a bot-protection challenge, Spidra tries to solve it and waits for the real page to load. If the challenge is still on screen after that, the job fails with an error saying so, and you are not charged. A challenge screen is never returned as the page content.

result.stats

Timing and usage information for the job.

Failed Jobs

When status is failed, the error field contains the reason:

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

jobId
string
required

The job ID returned from POST /scrape

Response

Job status and results

status
enum<string>

Current status of the scrape job

Available options:
waiting,
active,
completed,
failed,
delayed
progress
object
result
object | null

Present only when status is 'completed'

error
string | null

Error message if status is 'failed'