Webhooks

Two things can call your server: a crawl as it runs, and a monitor when something changed. Both send a POST with a JSON body to a public https URL you give them.

Crawl webhooks

Set webhookUrl when you start a crawl:

{ "baseUrl": "https://books.toscrape.com", "crawlInstruction": "Individual book pages", "maxPages": 10, "webhookUrl": "https://example.com/hooks/spidra" }

Event	When	Body
`crawl.page`	Each page is done	`{ "event": "crawl.page", "jobId": "...", "page": { "url": "...", "title": "...", "data": ... } }`
`crawl.completed`	The crawl finished	`{ "event": "crawl.completed", "jobId": "...", "pagesScraped": 10, "creditsUsed": 12 }`
`crawl.failed`	The crawl failed	`{ "event": "crawl.failed", "jobId": "...", "error": "..." }`

Crawl webhooks are sent once, with a 5-second timeout and no retries, and aren’t signed. Treat them as a signal: confirm the result with GET /crawl/{jobId} before you rely on it.

Monitor webhooks

Add a webhook to a monitor’s notify:

{ "notify": { "webhook": { "url": "https://example.com/hooks/spidra", "secret": "a-long-random-string" } } }

Each check that alerts sends one POST with the header X-Spidra-Event: monitor.check.completed. This is the body Send a Test Notification sends:

{
  "event": "monitor.check.completed",
  "monitor": { "id": "test", "name": "Test monitor", "alertOn": "new" },
  "check": { "id": "test", "summary": "Test notification", "url": "https://app.spidra.io/dashboard/monitors/test" },
  "judgment": null,
  "sources": [
    {
      "id": "test",
      "label": "Example",
      "added": [{ "title": "This is what a new item looks like", "url": "https://spidra.io" }],
      "changed": [{ "key": "x", "item": { "title": "And a changed one" }, "fields": [{ "field": "price", "previous": 30, "current": 25 }] }],
      "removed": [],
      "pages": []
    }
  ],
  "sentAt": "2026-10-24T18:50:30.112Z"
}

In a real alert, monitor and check carry your monitor’s and check’s IDs and check.url opens the check in the dashboard. added, changed, removed and pages are described in Get a Monitor Check.

Verify the signature

Monitor webhooks are signed. X-Spidra-Signature is sha256= followed by the HMAC-SHA256 of the raw body, keyed with the monitor’s secret. If you don’t set a secret, Spidra creates one; read it from Get a Monitor under notify.webhook.secret.

import crypto from "node:crypto";

export function isFromSpidra(rawBody, signature, secret) {
  const expected = "sha256=" + crypto.createHmac("sha256", secret).update(rawBody).digest("hex");
  return Boolean(signature) && signature.length === expected.length &&
    crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected));
}

import hashlib, hmac

def is_from_spidra(raw_body: bytes, signature: str, secret: str) -> bool:
    expected = "sha256=" + hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(signature or "", expected)

Compute it over the raw request body, before parsing the JSON. Re-serialising changes the bytes and the signature won’t match.

Your endpoint

Use https on a public address. Private and internal addresses are refused when you save the URL.
Answer with any 2xx quickly and do slow work afterwards.
Monitor webhooks are retried once, 2 seconds later, if your server answers 429, a 5xx or can’t be reached. Crawl webhooks aren’t retried.
Expect the occasional duplicate, and use jobId or check.id to de-duplicate.

Using the API

Scrape

Batch scrape

Crawl

Map

Search

Parse

Templates

Presets

Monitors

Logs

Account

Crawl webhooks

Monitor webhooks

Verify the signature

Your endpoint

​Crawl webhooks

​Monitor webhooks

​Verify the signature

​Your endpoint

Crawl webhooks

Monitor webhooks

Verify the signature

Your endpoint