Bun

The Spidra Bun integration (spidra-bun) brings web data intelligence to Bun’s ultra-fast native runtime:

Bun Native: Optimized for Bun’s native Fetch, Web Crypto, and standard Web APIs.
Bun.file() Document Parsing: parseBunFile() parses local PDFs, Word files, and spreadsheets directly from BunFile handles.
Bun.serve Handler: createSpidraBunHandler() provides a plug-and-play HTTP server handler.
HMAC Webhook Verification: verifyBunWebhook() verifies asynchronous job completion signatures with Bun’s built-in crypto.
Full API Surface: Scrape, Crawl, SERP Search, Batch Scraping, Document Parsing, Domain Mapping, Turnkey Templates, Presets, and Monitors.

Installation

bun add spidra-bun spidra

Get your API key from app.spidra.io under API Keys. Store it in .env or set SPIDRA_API_KEY.

Configuration

Set environment variables in .env:

SPIDRA_API_KEY=spd_live_your_api_key
SPIDRA_WEBHOOK_SECRET=whsec_your_signing_secret

# Optional overrides
SPIDRA_BASE_URL=https://api.spidra.io/api
SPIDRA_MAX_RETRIES=3
SPIDRA_BACKOFF_FACTOR=1.0

createBunSpidra() automatically reads Bun.env.SPIDRA_API_KEY and process.env.SPIDRA_API_KEY.

Quickstart

1. Basic Client Usage

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

const result = await spidra.scrape({
  urls: [{ url: "https://bun.sh" }],
  prompt: "Extract the main headline and key performance metrics",
  output: "json",
});

console.log("Scraped data:", result.data?.[0]?.content);

2. Document Parsing with `Bun.file()`

Parse local PDFs or office documents in one line using Bun.file():

import { createBunSpidra, parseBunFile } from "spidra-bun";

const spidra = createBunSpidra();
const file = Bun.file("./financial-report.pdf");

const doc = await parseBunFile(spidra, file, {
  prompt: "Extract quarterly revenue, operating expenses, and net profit",
});

console.log(doc.markdown);

3. Native Bun HTTP Server (`Bun.serve`)

import { createSpidraBunHandler } from "spidra-bun";

// Mounts /spidra/scrape, /crawl, /search, /batch, /map, /account, /webhooks
export default {
  port: 3000,
  fetch: createSpidraBunHandler({
    webhookSecret: process.env.SPIDRA_WEBHOOK_SECRET,
    onWebhookEvent: async (event) => {
      console.log(`Verified webhook event: ${event.event}`);
    },
  }),
};

Complete Spidra Features Guide

1. Scraping with Browser Actions & Proxies

Handle SPAs, bypass anti-bot shields with residential proxies, and interact with pages before extraction:

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

const scrapeResult = await spidra.scrape({
  urls: [{ url: "https://example.com/interactive-dashboard" }],
  prompt: "Extract all filtered table rows",
  output: "json",
  schema: {
    type: "object",
    properties: {
      metrics: {
        type: "array",
        items: {
          type: "object",
          properties: {
            name: { type: "string" },
            value: { type: "number" },
          },
        },
      },
    },
  },
  actions: [
    { type: "click", selector: "button#load-more" },
    { type: "wait", ms: 2000 },
    { type: "scroll", distance: 1000 },
  ],
  proxy: {
    type: "residential",
    country: "US",
  },
  waitForSelector: "table.data-grid",
  screenshot: true,
});

2. Deep Site Crawling

Crawl entire domains with breadth-first or depth-first traversal, prefix filtering, and sitemaps:

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

const crawlJob = await spidra.crawl({
  url: "https://docs.bun.sh",
  maxDepth: 3,
  maxPages: 100,
  allowBackwardLinks: false,
  allowExternalLinks: false,
  includePaths: ["/api/**", "/guides/**"],
  excludePaths: ["/v0.1/**"],
  ignoreSitemap: false,
  webhook: {
    url: "https://my-bun-app.com/spidra/webhooks",
  },
});

console.log(`Crawl job started with ID: ${crawlJob.jobId}`);

3. Real-Time SERP Web Search

Query web, news, and image indexes to feed live context into AI applications:

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

const searchResults = await spidra.search({
  query: "latest bun 1.1 release features",
  sources: ["web", "news"],
  limit: 10,
  filter: {
    domain: ["bun.sh", "github.com"],
  },
});

for (const item of searchResults.results) {
  console.log(`- ${item.title}: ${item.url}`);
}

4. High-Throughput Batch Scraping

Scrape multiple independent pages simultaneously:

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

const batchJob = await spidra.batchScrape({
  urls: [
    "https://news.ycombinator.com",
    "https://github.com/trending",
    "https://lobste.rs",
  ],
  prompt: "Extract top 5 trending stories with score and comment count",
  output: "json",
  webhook: {
    url: "https://my-bun-app.com/spidra/webhooks",
  },
});

console.log(`Batch job queued: ${batchJob.jobId}`);

5. Fast Domain Link Mapping

Discover all discoverable URLs on a domain without downloading full page bodies:

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

const siteMap = await spidra.map({
  url: "https://bun.sh",
  limit: 500,
  search: "package-manager",
});

console.log(`Discovered ${siteMap.links.length} matching URLs:`, siteMap.links);

6. Document Parsing

Extract clean markdown and tables from local files using Bun handles or online document URLs:

import { createBunSpidra, parseBunFile } from "spidra-bun";

const spidra = createBunSpidra();

// 1. Local file via Bun.file()
const localResult = await parseBunFile(spidra, Bun.file("./specs.pdf"), {
  prompt: "Summarize system architecture and performance benchmarks",
});

// 2. Remote document via URL
const remoteResult = await spidra.parse({
  url: "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
});

7. Turnkey AI Templates & Scheduled Monitors

Execute pre-tuned extraction templates and automate recurring scrapes:

import { createBunSpidra } from "spidra-bun";

const spidra = createBunSpidra();

// Run pre-tuned e-commerce template
const productData = await spidra.executeTemplate("ecommerce-product", {
  url: "https://store.example.com/item/123",
});

// Schedule recurring monitor
const monitor = await spidra.createMonitor({
  url: "https://status.example.com",
  cron: "0 */6 * * *",
  prompt: "Extract any ongoing incidents or degraded services",
  webhook: {
    url: "https://my-bun-app.com/spidra/webhooks",
  },
});

HMAC Webhook Verification

Verify webhook integrity with Bun’s native Web Crypto API:

import { verifyBunWebhook } from "spidra-bun";

export default {
  port: 3000,
  async fetch(req: Request) {
    if (req.method === "POST" && new URL(req.url).pathname === "/webhooks") {
      const signature = req.headers.get("x-spidra-signature") || "";
      const bodyText = await req.text();

      const isValid = await verifyBunWebhook(
        bodyText,
        signature,
        process.env.SPIDRA_WEBHOOK_SECRET!
      );

      if (!isValid) {
        return new Response("Invalid signature", { status: 401 });
      }

      const payload = JSON.parse(bodyText);
      console.log(`Processing verified event: ${payload.event}`);

      return new Response("OK", { status: 200 });
    }

    return new Response("Not Found", { status: 404 });
  },
};

Typed Error Handling

Catch and handle granular Spidra errors:

import {
  createBunSpidra,
  SpidraError,
  RateLimitError,
  TimeoutError,
  AuthenticationError,
} from "spidra-bun";

const spidra = createBunSpidra();

try {
  const result = await spidra.scrape({
    urls: [{ url: "https://example.com" }],
    prompt: "Extract page title",
  });
} catch (error) {
  if (error instanceof RateLimitError) {
    console.error(`Rate limited. Reset in ${error.retryAfter}s`);
  } else if (error instanceof AuthenticationError) {
    console.error("Invalid API key or unauthorized");
  } else if (error instanceof TimeoutError) {
    console.error("Extraction timed out");
  } else if (error instanceof SpidraError) {
    console.error(`Spidra API error [${error.status}]: ${error.message}`);
  } else {
    console.error("Unexpected error:", error);
  }
}

Fastify SDK

High-performance Spidra plugin for Fastify backends.

Remix SDK

Spidra loaders and actions for Remix and React Router.

SDKs

AI tools

Installation

Configuration

Quickstart

1. Basic Client Usage

2. Document Parsing with `Bun.file()`

3. Native Bun HTTP Server (`Bun.serve`)

Complete Spidra Features Guide

1. Scraping with Browser Actions & Proxies

2. Deep Site Crawling

3. Real-Time SERP Web Search

4. High-Throughput Batch Scraping

5. Fast Domain Link Mapping

6. Document Parsing

7. Turnkey AI Templates & Scheduled Monitors

HMAC Webhook Verification

Typed Error Handling

Fastify SDK

Remix SDK

Installation

Configuration

Quickstart

1. Basic Client Usage

2. Document Parsing with Bun.file()

3. Native Bun HTTP Server (Bun.serve)

Complete Spidra Features Guide

1. Scraping with Browser Actions & Proxies

2. Deep Site Crawling

3. Real-Time SERP Web Search

4. High-Throughput Batch Scraping

5. Fast Domain Link Mapping

6. Document Parsing

7. Turnkey AI Templates & Scheduled Monitors

HMAC Webhook Verification

Typed Error Handling

Related Frameworks

Fastify SDK

Remix SDK

2. Document Parsing with `Bun.file()`

3. Native Bun HTTP Server (`Bun.serve`)