Skip to main content
POST
Re-Extract from Existing Crawl
This does not re-crawl the website. Spidra reads the HTML and markdown already saved from the original crawl.

Prerequisites

The source crawl job must have a completed status before you call this endpoint. Calling /extract on a job that is still running, pending, or failed will return a 400 Bad Request.Poll GET /crawl/{jobId} and wait for "status": "completed" before proceeding.

Request Body

How It Works

  1. Pass the jobId of a completed crawl job. If you ran the crawl previously, this is the id field shown in your crawl history.
  2. Provide a transformInstruction describing what you want to extract.
  3. Spidra loads the saved content for each page and runs your prompt against it.
  4. A new crawl job is created with the results, which you can poll and download the same way as any other job.

When to Use This

  • You want to extract different fields from pages you already crawled
  • Your first extraction prompt wasn’t quite right and you want to try again
  • You need the same pages in two different formats, like JSON and CSV

Polling Results

The response returns a new jobId. Use the standard crawl endpoints to check progress and get results:

Common Errors

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

jobId
string
required

The ID of the completed source crawl job to extract from

Body

application/json
transformInstruction
string
required

Extraction prompt to apply to all pages from the source crawl. Maximum 5,000 characters.

Maximum string length: 5000

Response

Extraction job queued

status
string
Example:

"queued"

jobId
string

New job ID to poll for results

message
string