> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Authenticated Scraping

> Scrape content behind login walls using session cookies

Some websites require you to be logged in to see certain content. Authenticated scraping lets you access this content by providing your session cookies to Spidra.

***

## When You Need It

Use authenticated scraping when:

* The content is behind a login page
* You see different data when logged in vs logged out
* The website requires a subscription or account
* You want to scrape your own account data

***

## How It Works

1. Log into the target website in your browser
2. Copy your session cookies
3. Include them in your API request
4. Spidra uses those cookies to access the page as if it were you

***

## Getting Your Cookies

### Step 1: Log In

Open the website in Chrome, Firefox, or Edge and log into your account.

### Step 2: Open DevTools

Press `F12` or right-click → Inspect → go to the **Application** tab (Chrome) or **Storage** tab (Firefox).

### Step 3: Find Cookies

Click **Cookies** in the left sidebar, then select the website domain.

### Step 4: Copy Cookies

You have two options:

**Option A - Copy specific cookies:**
Look for authentication cookies (often named `session`, `auth`, `token`, or similar). Copy the name and value of each.

**Option B - Copy all cookies:**
Select all rows (Ctrl/Cmd+A), then copy (Ctrl/Cmd+C).

***

## Cookie Formats

Spidra accepts cookies in two formats. The API auto-detects which format you're using.

### Standard Format

The traditional `name=value` format, separated by semicolons:

```json theme={null}
{
  "urls": [{"url": "https://app.example.com/dashboard"}],
  "prompt": "Extract my account details",
  "output": "json",
  "cookies": "session_id=abc123xyz; auth_token=eyJhbGciOiJ..."
}
```

### Raw DevTools Format

Paste directly from Chrome DevTools without any formatting:

```json theme={null}
{
  "urls": [{"url": "https://app.example.com/dashboard"}],
  "prompt": "Extract my account details",
  "output": "json",
  "cookies": "session_id\tabc123xyz\t.example.com\t/\t2026-12-31\t50\t✓\t✓\tLax\nauth_token\teyJhbGciOiJ...\t.example.com\t/\t2026-12-31\t200\t✓\t✓\tStrict"
}
```

The API extracts the cookie name, value, domain, and path from each row. Extra columns like Size, HttpOnly, and SameSite are ignored.

***

## Examples

### Scraping a dashboard

```json theme={null}
{
  "urls": [{"url": "https://analytics.example.com/reports"}],
  "prompt": "Extract the monthly traffic summary",
  "output": "json",
  "cookies": "session=eyJ...; csrf_token=abc123"
}
```

### Crawling authenticated pages

```json theme={null}
{
  "baseUrl": "https://app.example.com/projects",
  "crawlInstruction": "Find all project pages",
  "transformInstruction": "Extract project name, status, and last updated date",
  "maxPages": 20,
  "cookies": "auth=eyJ...; user_id=12345"
}
```

***

## Tips

* **Session expiry**: Cookies expire. If your scrape fails with authentication errors, get fresh cookies.

* **Required cookies**: You usually don't need all cookies. Look for ones with names like `session`, `auth`, `token`, `jwt`, or `user`.

* **Domain matching**: Cookies are domain-specific. Make sure you're copying cookies from the correct domain.

* **Incognito test**: Before scraping, try opening the URL in an incognito window with your cookies to verify they work.

***

## Legal Responsibility

<Warning>
  You are responsible for ensuring your authenticated scraping complies with applicable laws and the website's Terms of Service. Only scrape content you're authorized to access. Cookies are processed transiently and never stored by Spidra.
</Warning>

***

<CardGroup cols={2}>
  <Card title="Submit a Scrape Job" icon="paper-plane" href="/api-reference/scraping/scrape">
    API reference
  </Card>

  <Card title="Submit a Crawl Job" icon="spider" href="/api-reference/crawling/crawl">
    Crawl with authentication
  </Card>
</CardGroup>