Spidra MCP Server

The Spidra MCP server gives your client access to web scraping, structured extraction, batch processing, crawling, browser actions, job history, and usage data as native MCP tools. The server is open source and available on GitHub.

Quickstart

The easiest way to use Spidra is through the hosted MCP server. There is nothing to install or keep running.

https://mcp.spidra.io/mcp

Add this URL to your MCP client. If the client supports OAuth, it will prompt you to sign in to Spidra and approve access. For automated environments, CI, or clients where you prefer a static credential, you can use a Spidra API key instead. Create an API key in Settings → API Keys at app.spidra.io, then send it with each request:

Authorization: Bearer spd_YOUR_API_KEY

The hosted MCP server uses your Spidra account and consumes credits in the same way as direct API requests. There is no separate keyless usage tier.

Connect your client

Pick your client below for exact setup steps. Every client can sign in with OAuth in one click, or use a static API key if you’d rather not.

Run:

claude mcp add --transport http spidra https://mcp.spidra.io/mcp

Start Claude Code and run /mcp if you need to complete authentication or check the connection.If you want to use an API key instead:

claude mcp add --transport http spidra https://mcp.spidra.io/mcp \
  --header "Authorization: Bearer spd_YOUR_API_KEY"

Open Customize → Connectors, click +, then select Add custom connector.Enter:

https://mcp.spidra.io/mcp

Click Add. Claude will guide you through authentication when you connect.If you need to use a static API key with Claude Desktop, use the local setup described in Run Spidra locally.

Add Spidra to your MCP configuration:

{
  "mcpServers": {
    "spidra": {
      "url": "https://mcp.spidra.io/mcp"
    }
  }
}

Use .cursor/mcp.json for a project-specific configuration, or your global Cursor MCP configuration if you want Spidra available across projects.Cursor will handle authentication when the server requires it.

Add Spidra to .vscode/mcp.json:

{
  "servers": {
    "spidra": {
      "type": "http",
      "url": "https://mcp.spidra.io/mcp"
    }
  }
}

VS Code will handle the OAuth flow when authentication is required.

Open Windsurf Settings → Cascade → MCP Servers, or edit the raw MCP configuration at:

~/.codeium/windsurf/mcp_config.json

Add:

{
  "mcpServers": {
    "spidra": {
      "serverUrl": "https://mcp.spidra.io/mcp"
    }
  }
}

If you want to use an API key instead, add an Authorization header:

{
  "mcpServers": {
    "spidra": {
      "serverUrl": "https://mcp.spidra.io/mcp",
      "headers": {
        "Authorization": "Bearer spd_YOUR_API_KEY"
      }
    }
  }
}

Open the MCP manager from the agent panel and select View raw config, or edit:

~/.gemini/config/mcp_config.json

For a workspace-specific configuration, use:

.agents/mcp_config.json

Add:

{
  "mcpServers": {
    "spidra": {
      "serverUrl": "https://mcp.spidra.io/mcp"
    }
  }
}

Antigravity can handle OAuth automatically for compatible remote MCP servers.If you want to use an API key instead:

{
  "mcpServers": {
    "spidra": {
      "serverUrl": "https://mcp.spidra.io/mcp",
      "headers": {
        "Authorization": "Bearer spd_YOUR_API_KEY"
      }
    }
  }
}

Add the hosted server:

codex mcp add spidra --url https://mcp.spidra.io/mcp

Then authenticate:

codex mcp login spidra

For API key authentication, keep the key in an environment variable:

export SPIDRA_API_KEY=spd_YOUR_API_KEY

Then add Spidra with that variable as the bearer token:

codex mcp add spidra \
  --url https://mcp.spidra.io/mcp \
  --bearer-token-env-var SPIDRA_API_KEY

Add Spidra as a remote HTTP server:

copilot mcp add --transport http spidra https://mcp.spidra.io/mcp

To use an API key, add the authorization header:

copilot mcp add --transport http \
  --header "Authorization: Bearer spd_YOUR_API_KEY" \
  spidra https://mcp.spidra.io/mcp

GitHub Copilot CLI stores user-level MCP configuration in:

~/.copilot/mcp-config.json

For OpenCode v2, add Spidra under mcp.servers:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "servers": {
      "spidra": {
        "type": "remote",
        "url": "https://mcp.spidra.io/mcp"
      }
    }
  }
}

OpenCode supports OAuth for remote servers by default. When Spidra requires authentication, connect through the OpenCode interface or run the MCP authentication command for the server.To use an API key instead, store it in an environment variable:

export SPIDRA_API_KEY=spd_YOUR_API_KEY

Then disable OAuth for the server and pass the key as a header:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "servers": {
      "spidra": {
        "type": "remote",
        "url": "https://mcp.spidra.io/mcp",
        "oauth": false,
        "headers": {
          "Authorization": "Bearer {env:SPIDRA_API_KEY}"
        }
      }
    }
  }
}

For additional clients and configuration options, see the full MCP reference.

What you can do

The Spidra MCP server exposes tools for common scraping and crawling workflows.

Scrape pages: extract clean Markdown or structured JSON from one or more URLs.
Process URLs in batches: run the same extraction across up to 50 URLs in parallel while keeping each result separate.
Crawl websites: start from one URL and describe which links Spidra should follow.
Interact before scraping: click, type, scroll, and loop over page elements before extracting content.
Use residential proxies: access geo-restricted or bot-protected pages when your Spidra configuration allows it.
Inspect previous jobs: retrieve scrape and crawl history without starting the work again.
Check usage: inspect request, credit, and token usage from your MCP client.

Available tools

Tool	Description
`spidra_scrape`	Scrape 1 to 3 URLs and return the extracted result
`spidra_check_scrape_status`	Check a scrape that exceeded its initial wait window
`spidra_batch_scrape`	Process 2 to 50 URLs in parallel
`spidra_check_batch_status`	Check batch progress and retrieve completed results
`spidra_cancel_batch`	Cancel a running batch
`spidra_crawl`	Crawl a site from a starting URL
`spidra_check_crawl_status`	Check crawl progress and completed results
`spidra_crawl_pages`	Retrieve individual crawl pages and download links
`spidra_crawl_extract`	Run a new extraction over an existing completed crawl
`spidra_cancel_crawl`	Cancel a running crawl
`spidra_scrape_logs`	Look up previous jobs and their results
`spidra_usage`	Check request, credit, and token usage

For complete parameters, schemas, examples, and tool-selection guidance, see the full MCP reference.

Run Spidra locally

You can also run the MCP server on your own machine using the standard input/output transport. This is useful when your client does not support remote MCP servers, when you want to manage the process locally, or when you need to point the server at a different Spidra API environment.

Run with npx

No global installation is required:

env SPIDRA_API_KEY=spd_YOUR_API_KEY npx -y spidra-mcp

Install globally

npm install -g spidra-mcp

Then start the server from any MCP client that supports local stdio servers. For example, Claude Code:

claude mcp add spidra \
  -e SPIDRA_API_KEY=spd_YOUR_API_KEY \
  -- npx -y spidra-mcp

Start a new Claude Code session and run /mcp to confirm that Spidra is connected.

Self-host over Streamable HTTP

The package can also run as a Streamable HTTP server. Use this when you want to host your own MCP endpoint, place it behind your own network controls, or point it at a different Spidra API environment.

env HTTP_STREAMABLE_SERVER=true \
  SPIDRA_API_KEY=spd_YOUR_API_KEY \
  npx -y spidra-mcp

The default endpoint is:

http://localhost:3000/mcp

If the server needs to serve multiple users, omit SPIDRA_API_KEY from the server environment and send the appropriate API key with each request instead:

Authorization: Bearer spd_YOUR_API_KEY

Use Spidra with n8n

You can connect Spidra directly to an n8n AI Agent through the MCP Client Tool node.

Add an AI Agent node to your workflow.
Add an MCP Client Tool to the agent.
Set the MCP endpoint to:

https://mcp.spidra.io/mcp

Choose the Streamable HTTP transport.
Configure authentication with your Spidra API key as a bearer token.
Choose whether to expose all Spidra tools or only the tools your workflow needs.

Use:

spd_YOUR_API_KEY

as the bearer token value. If n8n is connecting to a self-hosted Spidra MCP server, make sure the MCP endpoint is reachable from the machine or container where n8n runs.

Configuration

These environment variables apply when you run the MCP server yourself.

Variable	Required	Description
`SPIDRA_API_KEY`	Usually	Spidra API key used by the local server. It can be omitted for HTTP mode when every request supplies its own key.
`SPIDRA_API_URL`	No	Overrides the Spidra API base URL. Useful for staging or custom environments.
`HTTP_STREAMABLE_SERVER`	No	Set to `true` to run the Streamable HTTP transport instead of stdio.
`PORT`	No	Port used by the self-hosted HTTP server. Defaults to `3000`.
`HOST`	No	Host interface used by the HTTP server. Defaults to `localhost`.

For HTTP connections, you can also send the API key with each request:

Authorization: Bearer spd_YOUR_API_KEY

If a request includes both an API key header and an authenticated OAuth session, Spidra uses the API key from the request.

Error handling

Spidra returns API failures as MCP tool errors rather than crashing the server. Error messages are designed to tell the client what to do next:

Rate-limit errors indicate when the request can be retried.
Validation errors identify the input that needs to change.
Permanent errors explain when retrying the same request will not help.
Long-running scrape operations return a job ID that can be checked with spidra_check_scrape_status.

Example:

{
  "content": [
    {
      "type": "text",
      "text": "Spidra API error 404 (JOB_NOT_FOUND): Scrape not found. Do not retry with the same input."
    }
  ],
  "isError": true
}

Development

The MCP server is open source:

git clone https://github.com/spidra-io/spidra-mcp-server.git
cd spidra-mcp-server
npm install

Build the project:

npm run build

Run the test suite:

npm test

Run the type checker:

npm run typecheck

The test suite starts the built MCP server as a real stdio process and communicates with it over JSON-RPC against a fake Spidra API, so the tests do not consume Spidra credits. Development requires Node.js 20 or newer. The project is MIT licensed. Pull requests are welcome.

Full MCP Reference

Client configuration, parameters, tool examples, troubleshooting, and detailed reference material.

SDKs Overview

Prefer working in code? Explore the official Spidra SDKs.

Get Started

Core Features

Advanced

Dashboard

Billing & Usage

Integrations

Spidra MCP Server

Quickstart

Connect your client

What you can do

Available tools

Run Spidra locally

Run with npx

Install globally

Self-host over Streamable HTTP

Use Spidra with n8n

Configuration

Error handling

Development

Full MCP Reference

SDKs Overview

​Quickstart

​Connect your client

​What you can do

​Available tools

​Run Spidra locally

​Run with npx

​Install globally

​Self-host over Streamable HTTP

​Use Spidra with n8n

​Configuration

​Error handling

​Development

Full MCP Reference

SDKs Overview

Quickstart

Connect your client

What you can do

Available tools

Run Spidra locally

Run with npx

Install globally

Self-host over Streamable HTTP

Use Spidra with n8n

Configuration

Error handling

Development