> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured output

> Add a JSON Schema and get the same field names and types from every page: numbers as numbers, nulls for anything missing, lists as arrays.

With only a `prompt`, the AI decides the shape of the answer: field names, types and nesting can change between pages and runs, and prices come back as text like `"£51.77"`. Add a `schema` and every result has exactly the fields and types you define. Use one whenever code reads the result.

Schemas work on [Scrape](https://docs.spidra.io/scrape.md), [Batch scrape](https://docs.spidra.io/batch-scrape.md), [Crawl](https://docs.spidra.io/crawl.md) and [Parse](https://docs.spidra.io/parse.md).

## A first schema

```json theme={null}
{
  "urls": [{ "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }],
  "prompt": "Extract the book details",
  "schema": {
    "type": "object",
    "properties": {
      "title": { "type": "string" },
      "price": { "type": "number" },
      "currency": { "type": ["string", "null"] },
      "in_stock": { "type": ["boolean", "null"] },
      "rating": { "type": ["string", "null"] }
    }
  }
}
```

`result.content`:

```json theme={null}
{ "title": "A Light in the Attic", "price": 51.77, "currency": "£", "in_stock": true, "rating": null }
```

* `price` is the number `51.77`; the `£` moved into `currency`.
* `in_stock` is a real boolean, not the text "In stock".
* `rating` is `null`: the site shows its stars as an image, and Spidra reads the page's text.

It cost 2 credits: 1 for the page and 1 for the AI.

## Every field comes back

Every field in your schema is in the result. When the page doesn't have a value, the field is `null`; it's never left out. On a book page with no ISBN:

```json theme={null}
{
  "type": "object",
  "properties": {
    "title":  { "type": "string" },
    "price":  { "type": "number" },
    "isbn":   { "type": ["string", "null"] },
    "author": { "type": ["string", "null"] }
  }
}
```

```json theme={null}
{ "title": "A Light in the Attic", "price": 51.77, "isbn": null, "author": "Shel Silverstein" }
```

The answer was the same whether `required` listed the fields or not, so code can always read every key. Give fields that may be missing a nullable type, as below, so `null` is a valid value.

## Nullable fields

Write the type as a list that includes `"null"`:

```json theme={null}
{ "type": ["string", "null"] }
```

This works for every type: `["number", "null"]`, `["boolean", "null"]`, `["array", "null"]`.

## Lists of items

The root of a schema is always an object. To get a list, put an array inside it:

```json theme={null}
{
  "urls": [{ "url": "https://books.toscrape.com/catalogue/category/books/mystery_3/index.html" }],
  "prompt": "Every book on the page",
  "schema": {
    "type": "object",
    "properties": {
      "books": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "title": { "type": "string" },
            "price": { "type": "number" },
            "in_stock": { "type": ["boolean", "null"] }
          }
        }
      }
    }
  }
}
```

That returned all 20 books on the page. The first three:

```json theme={null}
{
  "books": [
    { "title": "Sharp Objects", "price": 47.82, "in_stock": true },
    { "title": "In a Dark, Dark Wood", "price": 19.63, "in_stock": true },
    { "title": "The Past Never Ends", "price": 56.5, "in_stock": true }
  ]
}
```

A flat schema (no array) on a list page returns one item. In a [crawl](https://docs.spidra.io/crawl.md), the schema runs on every returned page, so write it for one page of the kind you asked for.

## Fixed choices with `enum`

`enum` limits a field to the values you list. Include `null` for "none of these":

```json theme={null}
{
  "type": "object",
  "properties": {
    "title": { "type": "string" },
    "stars": { "type": ["string", "null"], "enum": ["one", "two", "three", "four", "five", null] },
    "availability": { "type": ["string", "null"], "enum": ["in_stock", "out_of_stock", null] }
  }
}
```

```json theme={null}
{ "title": "A Light in the Attic", "stars": null, "availability": "in_stock" }
```

`stars` is `null` because the rating isn't in the page's text.

## Nested objects

Define `properties` inside a property:

```json theme={null}
{
  "type": "object",
  "properties": {
    "title": { "type": "string" },
    "price": {
      "type": "object",
      "properties": {
        "amount":   { "type": ["number", "null"] },
        "currency": { "type": ["string", "null"] }
      }
    }
  }
}
```

```json theme={null}
{ "title": "A Light in the Attic", "price": { "amount": 51.77, "currency": "£" } }
```

## Prompt and schema together

The schema sets the shape; the `prompt` tells the AI what to pick and how to write it. Here the prompt filters the quotes and lowercases the tags:

```json theme={null}
{
  "urls": [{ "url": "https://quotes.toscrape.com/" }],
  "prompt": "Only the quotes by Albert Einstein. Give the tags in lowercase.",
  "schema": {
    "type": "object",
    "properties": {
      "quotes": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": { "text": { "type": "string" }, "tags": { "type": "array", "items": { "type": "string" } } }
        }
      }
    }
  }
}
```

```json theme={null}
{
  "quotes": [
    { "text": "The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.", "tags": ["change", "deep-thoughts", "thinking", "world"] },
    { "text": "There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.", "tags": ["inspirational", "life", "live", "miracle", "miracles"] }
  ]
}
```

(Shortened to two of the Einstein quotes on the page.)

## Generate a schema from Zod or Pydantic

Spidra takes standard JSON Schema, so you can generate it from the models you already have.

```typescript Zod theme={null}
import { z } from "zod";

const Book = z.object({
  title: z.string(),
  price: z.number(),
  in_stock: z.boolean().nullable(),
});

const result = await spidra.scrape({
  urls: [{ url: "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }],
  prompt: "Extract the book details",
  schema: z.toJSONSchema(Book),
});

const book = Book.parse(result.content);
```

```python Pydantic theme={null}
from typing import Optional
from pydantic import BaseModel
from spidra import ScrapeParams, ScrapeUrl

class Book(BaseModel):
    title: str
    price: float
    in_stock: Optional[bool] = None

result = client.scrape(ScrapeParams(
    urls=[ScrapeUrl(url="https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html")],
    prompt="Extract the book details",
    schema=Book.model_json_schema(),
))

book = Book.model_validate(result.content)
```

Validating the result with the same model gives you typed data end to end. There's also a visual [JSON Schema generator](https://spidra.io/tools/json-schema-generator).

## What a schema can contain

Supported keywords: `type` (including lists with `"null"`), `properties`, `required`, `items`, `enum`, `description` and `nullable`. A `description` on a field is a good place for hints ("price without the currency symbol").

Keywords Spidra can't use, such as `$ref`, `oneOf`, `allOf`, `not` and `if`, are ignored, and the submit response lists them in `schema_warnings`:

```json theme={null}
{
  "status": "queued",
  "jobId": "536b5dd6-84e7-4287-b514-5c7250750695",
  "schema_warnings": ["\"anyOf\" at schema.properties.isbn is not supported and will be ignored"]
}
```

The job still runs. Pydantic writes `Optional` fields as `anyOf` with `null`, which shows this warning; those fields still come back, as `null` when missing.

## Schema errors

A schema with a structural problem is refused with `422` before anything runs, and nothing is charged:

```json theme={null}
{
  "status": "error",
  "message": "Invalid schema. Fix the errors below and try again.",
  "errors": ["schema is too deeply nested (depth: 6). Maximum nesting depth is 5. Flatten your schema."]
}
```

* The root `type` must be `"object"`. Wrap lists in an object property, as above.
* Nesting goes 5 levels deep at most.
* The schema can be up to 10 KB.

## Sites that come back structured already

Links from [supported sites](https://docs.spidra.io/sites/overview.md) such as Amazon, Google Maps and LinkedIn have the site's own fields in `structuredData`, and with `"output": "json"` and no prompt or schema those fields are the `content`, with no AI. Add a schema only if you want to reshape them.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.