> ## Documentation Index
> Fetch the complete documentation index at: https://docs.spidra.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Laravel

> Official Laravel package for Spidra — AI web scraping, recursive crawling, SERP search, and document parsing with Facades, Service Providers, and verified webhook events.

The Spidra Laravel package (`spidra/spidra-laravel`) brings AI-powered web intelligence directly into Laravel:

* **Laravel Auto-Discovery**: Automatically registers `SpidraServiceProvider` and the `Spidra` Facade.
* **Config Publishing**: Configurable via `config/spidra.php` and `.env`.
* **`Spidra` Facade & `spidra()` Helper**: Ergonomic syntax for controllers, queue jobs, Artisan commands, and Blade templates.
* **Dependency Injection**: Inject `Spidra\SpidraClient` or `Spidra\Laravel\SpidraManager` anywhere in your application.
* **HMAC Webhook Routing & Events**: One-line `Route::spidraWebhooks()` macro with automatic HMAC signature verification and `SpidraWebhookReceived` event dispatching.
* **Full API Surface**: Web Scraping, Recursive Crawling, SERP Search, Batch Scraping, Document Parsing, Domain Mapping, Turnkey Templates, Presets, and Monitors.

## Installation

Install the package via Composer:

```bash theme={null}
composer require spidra/spidra-laravel spidra/spidra-php
```

Publish the configuration file:

```bash theme={null}
php artisan vendor:publish --tag=spidra-config
```

This creates `config/spidra.php`.

> [!NOTE]
>
> Get your API key from [app.spidra.io](https://app.spidra.io) under **API Keys**.
> Store it in your `.env` file.

## Configuration

Set your Spidra credentials in `.env`:

```bash theme={null}
SPIDRA_API_KEY=spd_live_your_api_key_here
SPIDRA_WEBHOOK_SECRET=whsec_your_webhook_signing_secret

# Optional overrides:
SPIDRA_BASE_URL=https://api.spidra.io/api
SPIDRA_TIMEOUT=30
SPIDRA_WEBHOOK_ROUTE=spidra/webhooks
```

## Quickstart

### 1. In a Controller using the Facade

```php theme={null}
namespace App\Http\Controllers;

use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Spidra\Laravel\Facades\Spidra;

class ScrapeController extends Controller
{
    public function __invoke(Request $request): JsonResponse
    {
        $validated = $request->validate([
            'url' => ['required', 'url'],
        ]);

        $data = Spidra::scrape([
            'urls'   => [['url' => $validated['url']]],
            'prompt' => 'Extract product title, price, in_stock status, and top customer review',
            'output' => 'json',
        ]);

        return response()->json($data);
    }
}
```

### 2. In a Route Closure

```php theme={null}
use Illuminate\Support\Facades\Route;
use Spidra\Laravel\Facades\Spidra;

Route::get('/search-news', function () {
    $results = Spidra::search([
        'query'   => 'Laravel 12 release features',
        'sources' => ['news', 'web'],
        'limit'   => 10,
    ]);

    return response()->json($results);
});
```

### 3. In a Queueable Job

```php theme={null}
namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Spidra\SpidraClient;

class CrawlDocumentationJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(public string $docsUrl)
    {
    }

    public function handle(SpidraClient $spidra): void
    {
        $crawl = $spidra->crawl([
            'url'          => $this->docsUrl,
            'maxDepth'     => 3,
            'maxPages'     => 100,
            'includePaths' => ['/docs/**', '/guide/**'],
            'webhook'      => [
                'url' => route('spidra.webhook'),
            ],
        ], wait: false);

        logger()->info("Crawl job queued: {$crawl['jobId']}");
    }
}
```

### 4. Using the `spidra()` Helper

```php theme={null}
$account = spidra()->getAccount();
$creditsRemaining = $account['credits']['remaining'] ?? 0;
```

## Complete Spidra Features Guide

### 1. Scraping with Browser Actions & Proxies

Handle JavaScript-heavy SPAs, dynamic tables, and login screens using residential proxies:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

$result = Spidra::scrape([
    'urls' => [['url' => 'https://example.com/interactive-pricing']],
    'prompt' => 'Extract plan names, monthly prices, and included feature list',
    'output' => 'json',
    'schema' => [
        'type' => 'object',
        'properties' => [
            'plans' => [
                'type' => 'array',
                'items' => [
                    'type' => 'object',
                    'properties' => [
                        'name' => ['type' => 'string'],
                        'price' => ['type' => 'number'],
                        'features' => [
                            'type' => 'array',
                            'items' => ['type' => 'string']
                        ]
                    ]
                ]
            ]
        ]
    ],
    'actions' => [
        ['type' => 'click', 'selector' => 'button#annual-toggle'],
        ['type' => 'wait', 'ms' => 1500],
        ['type' => 'scroll', 'distance' => 800],
    ],
    'proxy' => [
        'type' => 'residential',
        'country' => 'US',
    ],
    'waitForSelector' => 'div.pricing-tables',
    'screenshot' => true,
]);
```

### 2. Deep Site Crawling

Initiate recursive domain crawls with depth controls, path filters, and webhook callbacks:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

// Asynchronous crawl job
$crawl = Spidra::crawl([
    'url'                => 'https://laravel.com',
    'maxDepth'           => 3,
    'maxPages'           => 200,
    'allowBackwardLinks' => false,
    'allowExternalLinks' => false,
    'includePaths'       => ['/docs/11.x/**'],
    'excludePaths'       => ['/docs/10.x/**', '/docs/9.x/**'],
    'ignoreSitemap'      => false,
    'webhook'            => [
        'url' => url('/spidra/webhooks'),
    ],
], wait: false);

$jobId = $crawl['jobId'];

// Poll crawl status
$status = Spidra::client()->crawl->get($jobId);

// Download completed crawl pages
$pages = Spidra::client()->crawl->pages($jobId, page: 1, limit: 50);
```

### 3. Real-Time SERP Web Search

Query web, news, and images to ground AI models with fresh web data:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

$search = Spidra::search([
    'query'          => 'Laravel 12 best practices and updates',
    'sources'        => ['web', 'news'],
    'limit'          => 15,
    'timeRange'      => 'month',
    'country'        => 'US',
    'includeDomains' => ['laravel.com', 'laravel-news.com', 'github.com'],
]);

foreach ($search['results'] ?? [] as $item) {
    echo $item['title'] . ' - ' . $item['url'] . PHP_EOL;
}
```

### 4. High-Throughput Batch Scraping

Scrape multiple independent URLs concurrently in a single batch job:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

$batch = Spidra::batchScrape([
    'urls' => [
        'https://news.ycombinator.com',
        'https://github.com/trending',
        'https://lobste.rs',
    ],
    'prompt' => 'Extract top 5 trending posts with title, link, and score',
    'output' => 'json',
    'webhook' => [
        'url' => url('/spidra/webhooks'),
    ],
], wait: false);

$batchId = $batch['jobId'];
```

### 5. Fast Domain Link Mapping

Discover all discoverable URLs across a domain without downloading full page bodies:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

$map = Spidra::map([
    'url'               => 'https://laravel.com',
    'limit'             => 500,
    'search'            => 'octane',
    'includeSubdomains' => false,
]);

$links = $map['links'] ?? [];
```

### 6. Document Parsing (PDF, Word, Excel, PPTX)

Extract clean markdown and tables from remote URLs or local document files:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

// Parse a document via URL
$doc = Spidra::parse([
    'url'          => 'https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf',
    'mode'         => 'ocr', // 'auto' | 'fast' | 'ocr'
    'prompt'       => 'Extract executive summary and key numerical findings',
    'includePages' => true,
]);

echo $doc['markdown'] ?? '';
```

### 7. Turnkey AI Templates & Scheduled Monitors

Execute pre-tuned extraction templates and schedule automated website monitors:

```php theme={null}
use Spidra\Laravel\Facades\Spidra;

// 1. Run a pre-built template
$product = Spidra::runTemplate('ecommerce-product', [
    'url' => 'https://store.example.com/item/12345',
]);

// 2. Schedule recurring monitor
$monitor = Spidra::client()->monitors->create([
    'url'     => 'https://status.laravel.com',
    'cron'    => '0 */6 * * *',
    'prompt'  => 'Detect any degraded services or outages',
    'webhook' => [
        'url' => url('/spidra/webhooks'),
    ],
]);
```

## Webhook Handling & Laravel Events

Spidra sends asynchronous webhooks upon completion of crawl jobs, batch scrapes, or monitor alerts.

### 1. Register Webhook Route

In `routes/api.php` or `routes/web.php`:

```php theme={null}
use Illuminate\Support\Facades\Route;

// Registers POST /spidra/webhooks protected by VerifySpidraSignature middleware
Route::spidraWebhooks();

// Or specify a custom URL:
Route::spidraWebhooks('api/v1/spidra-events');
```

> [!NOTE]
>
> If registered inside `routes/web.php`, add `spidra/*` to the `$except` array in `app/Http/Middleware/VerifyCsrfToken.php` or `bootstrap/app.php` (Laravel 11+).

### 2. Listen to Webhook Events

When a verified webhook arrives, `SpidraWebhookReceived` is automatically dispatched.

Create a listener:

```bash theme={null}
php artisan make:listener HandleSpidraWebhook
```

In `app/Listeners/HandleSpidraWebhook.php`:

```php theme={null}
namespace App\Listeners;

use Spidra\Laravel\Events\SpidraWebhookReceived;

class HandleSpidraWebhook
{
    public function handle(SpidraWebhookReceived $event): void
    {
        $eventType = $event->event;
        $jobId = $event->jobId;
        $payload = $event->payload;

        logger()->info("Spidra Webhook [{$eventType}] received for job: {$jobId}");

        match ($eventType) {
            'crawl.completed' => $this->handleCrawlCompleted($jobId, $payload),
            'batch.completed' => $this->handleBatchCompleted($jobId, $payload),
            'monitor.alert'   => $this->handleMonitorAlert($jobId, $payload),
            default           => null,
        };
    }

    protected function handleCrawlCompleted(string $jobId, array $payload): void
    {
        // Process crawl results
    }

    protected function handleBatchCompleted(string $jobId, array $payload): void
    {
        // Process batch results
    }

    protected function handleMonitorAlert(string $jobId, array $payload): void
    {
        // Notify team via Slack, Email, etc.
    }
}
```

In `app/Providers/EventServiceProvider.php` (or auto-discovered in Laravel 11+):

```php theme={null}
use App\Listeners\HandleSpidraWebhook;
use Spidra\Laravel\Events\SpidraWebhookReceived;

protected $listen = [
    SpidraWebhookReceived::class => [
        HandleSpidraWebhook::class,
    ],
];
```

## Error Handling

All Spidra exceptions are strongly typed:

```php theme={null}
use Spidra\Exceptions\AuthenticationException;
use Spidra\Exceptions\InsufficientCreditsException;
use Spidra\Exceptions\RateLimitException;
use Spidra\Exceptions\ServerException;
use Spidra\Exceptions\SpidraException;
use Spidra\Laravel\Facades\Spidra;

try {
    $result = Spidra::scrape(['urls' => [['url' => 'https://example.com']]]);
} catch (AuthenticationException $e) {
    logger()->error("Invalid Spidra API key: {$e->getMessage()}");
} catch (RateLimitException $e) {
    logger()->warning("Rate limit exceeded. Retry later.");
} catch (InsufficientCreditsException $e) {
    logger()->error("Out of Spidra credits.");
} catch (ServerException $e) {
    logger()->error("Spidra API server error [{$e->getCode()}]: {$e->getMessage()}");
} catch (SpidraException $e) {
    logger()->error("Spidra error [{$e->status}]: {$e->getMessage()}");
}
```

## Testing & Mocking

Test your controllers and jobs using the `Spidra` facade mock:

```php theme={null}
namespace Tests\Feature;

use Spidra\Laravel\Facades\Spidra;
use Tests\TestCase;

class ProductScrapeTest extends TestCase
{
    public function test_scrapes_product_successfully(): void
    {
        Spidra::shouldReceive('scrape')
            ->once()
            ->with([
                'urls'   => [['url' => 'https://example.com/product/1']],
                'prompt' => 'Extract title and price',
                'output' => 'json',
            ])
            ->andReturn([
                'success' => true,
                'data' => [
                    ['content' => ['title' => 'Wireless Headphones', 'price' => 99.99]]
                ],
            ]);

        $response = $this->postJson('/api/scrape', [
            'url' => 'https://example.com/product/1',
        ]);

        $response->assertOk()
            ->assertJsonPath('data.0.content.title', 'Wireless Headphones');
    }
}
```

## Related SDKs

> [card] [PHP SDK](https://docs.spidra.io/sdks/php.md)
>
> Core PHP client library with Guzzle for vanilla PHP applications.

> [card] [FastAPI SDK](https://docs.spidra.io/sdks/fastapi.md)
>
> High-performance async Python integration with FastAPI.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.