# MCP Server

Connect Claude, ChatGPT, Cursor, Grok, or any MCP client to lobstr.io and run web scrapers in your account — discover, configure, run, monitor, and retrieve results, all in the conversation.

The lobstr.io **MCP Server** lets an AI client operate your lobstr.io account
directly. Discover a scraper, read its inputs, run it, watch the run, and pull
results — without leaving the chat and without manual CSV export/import.

**Endpoint:** `https://mcp.lobstr.io/mcp`

<CardGroup cols={3}>
  <Card title="22 Tools" icon="wrench">
    Discover scrapers, run and monitor them, fetch results, and check your
    credits and connected platform accounts.
  </Card>
  <Card title="OAuth" icon="lock">
    Connect with your own lobstr.io login. You approve on lobstr.io — no password
    ever reaches the AI client.
  </Card>
  <Card title="Streamable HTTP" icon="globe">
    Standard MCP transport, compatible with Claude, ChatGPT, Cursor, Grok, and Codex.
  </Card>
</CardGroup>

<Note>
  This is the **operational** server — it acts in your account and spends
  credits. For a public, no-auth assistant that only answers questions about the
  API docs, see the [Documentation MCP](https://staging.docs.lobstr.io/docs/docs-mcp).
</Note>

## Before you start

- A [lobstr.io account](https://app.lobstr.io) with credits — running a scraper
  spends credits from your balance.
- An MCP-capable client (Claude, ChatGPT, Cursor, Grok, Codex, …).

No client ID or secret to paste: the server supports Dynamic Client
Registration, so your client registers itself.

## Setup

<CodeGroup>
```bash Claude Code
claude mcp add --transport http lobstr https://mcp.lobstr.io/mcp
```

```json Claude Desktop
{
  "mcpServers": {
    "lobstr": {
      "type": "http",
      "url": "https://mcp.lobstr.io/mcp"
    }
  }
}
```

```json Cursor (.cursor/mcp.json)
{
  "mcpServers": {
    "lobstr": {
      "type": "http",
      "url": "https://mcp.lobstr.io/mcp"
    }
  }
}
```

```bash Codex CLI
codex mcp add lobstr --url https://mcp.lobstr.io/mcp
```

```toml Codex (~/.codex/config.toml)
[mcp_servers.lobstr]
url = "https://mcp.lobstr.io/mcp"
```
</CodeGroup>

**Claude.ai and ChatGPT** — add it as a **custom connector** in settings, using
`https://mcp.lobstr.io/mcp`. Both drive the sign-in flow for you.

**Codex** — `codex mcp add` starts the sign-in right away. If you added the
server through `config.toml`, or need to sign in again, run
`codex mcp login lobstr`.

**Grok** — available to Grok Bots with a SuperGrok plan or through a Cursor
subscription:

1. In any Bot, send: *Add a custom mcp server called lobstr.io, it's at
   https://mcp.lobstr.io/mcp* (keep the trailing `/mcp`).
2. When the Bot asks to add the lobstr.io connector, choose **Add it**.
3. Use the connect card the Bot posts to open the lobstr.io consent page, and click **Allow**.
4. Ask the Bot *Who am I?* to check the connection.

The server then shows for all your Bots under **Settings → Plugins → Yours**.
More in the [Grok guide](https://help.lobstr.io/integrations/grok).

After adding the server, your client opens the lobstr.io consent page: you're
already signed in, you click **Allow**, and you're returned to the client. On a
headless box the authorize URL is printed for you to open in a browser.

## Connecting your account

The server uses OAuth 2.1 with PKCE. You approve the connection on lobstr.io's
own site, so **no password is ever sent to the AI client or to the MCP server**.
You choose what to allow, per these scopes:

| Scope | Grants |
|---|---|
| `crawlers:read` | Discover scrapers and read their settings |
| `runs:read` | Check the status of your runs |
| `results:read` | Read your scraped results |
| `profile:read` | View your profile (name, email, plan) and credit balance |
| `account:read` | View your connected platform accounts (LinkedIn, Facebook, …) and their status |
| `runs:execute` | Run scrapers on your behalf (spends credits); together with `account:read`, also lets the client attach a connected account to a squid |

You can revoke access at any time from your lobstr.io account.

## Tools

A **crawler** is a scraper template (e.g. "Google Maps"); a **squid** is one you've
configured from it. A **run** is one execution of a squid.

#### Discover & inspect

| Tool | Type | Description |
|---|---|---|
| `search_scrapers` | read | Find the most relevant scrapers (crawlers) for a request. |
| `get_scraper_details` | read | A scraper's input schema, output fields, and pricing. |
| `list_my_scrapers` | read | Your saved scrapers (squids) — paged, with name / crawler filters. |
| `get_my_scraper` | read | Inspect one saved scraper's configuration. |

#### Run & monitor

| Tool | Type | Description |
|---|---|---|
| `run_scraper` | **write · spends credits** | Configure and start a run. Returns a `run_id`. |
| `get_run` | read | Status and progress of a run. |
| `wait_for_run` | read | Wait for a run to finish, up to 50 seconds per call; call again if it's still going. |
| `list_runs` | read | Recent runs for a scraper. |
| `abort_run` | write | Stop a running job. |
| `empty_scraper` | write | Clear a scraper's tasks so it can be reused with new inputs. |
| `deactivate_scraper` | write | Stop a saved scraper's schedule and any run in progress, without deleting it. |

#### Build a run step by step

An alternative to `run_scraper` for building up a squid across several calls
instead of one — useful for large or incremental task lists.

| Tool | Type | Description |
|---|---|---|
| `create_squid` | write | Create a saved scraper (squid) from a crawler, without running it, with an optional webhook. |
| `add_tasks` | write | Add input rows to a squid; callable repeatedly to build up a batch. |
| `update_scraper` | write | Change a squid's name, settings, concurrency, or webhook without running it or touching its tasks. |
| `estimate_run` | read | Credit cost, projected results, and estimated time for a squid's current tasks. |

#### Results

| Tool | Type | Description |
|---|---|---|
| `get_results` | read | One page of results for a run (row-capped, with field selection). |
| `get_results_url` | read | A signed URL to download the full result set as a file. |

#### Your account

| Tool | Type | Description |
|---|---|---|
| `check_credits` | read | Your credit balance, the slots your runs in progress use, and how many squids your plan holds. |
| `whoami` | read | Your account identity (email, name, plan). |

#### Connected accounts

The platform logins your scrapers authenticate with (LinkedIn, Facebook, …) —
distinct from your lobstr.io account above.

| Tool | Type | Description |
|---|---|---|
| `list_accounts` | read | The platform accounts scrapers log in with, and their status. |
| `get_account` | read | One connected account's status and which scrapers use it. |
| `attach_account` | write | Link a connected account to a squid so it can launch a run. See [account-backed scrapers](#account-backed-scrapers) below. |

`list_accounts` and `get_account` return **metadata only** — never credentials
or cookies. `attach_account` only ever sends account ids it looked up itself.

New scrapers work automatically — `get_scraper_details` reads each scraper's
live input schema, so nothing here needs updating when lobstr.io adds one.

## Credits and confirmation

Running a scraper spends credits, and lobstr.io prices **per result row** — so
the total isn't known until the run completes. `run_scraper` estimates the cost
and, when it's above a threshold or unknown, returns a confirmation request
instead of running; the client asks you, and only then does it execute. The read
tools never spend credits.

## Account-backed scrapers

Some scrapers log in to a platform on your behalf — the LinkedIn Leads Scraper
and Sales Navigator Leads Scraper, for example — and need a **connected
account** synced in your lobstr.io dashboard first. Without one, the run
starts but fails immediately with no account available to launch it.

- **The account type must match the scraper.** An account is linked to one
  platform (LinkedIn, Facebook, …); it can only run a scraper for that same
  platform.
- **`run_scraper` attaches one automatically** when exactly one connected
  account of the right type is healthy and not rate limited. With several
  matching accounts, or none, it asks you to choose instead of guessing —
  connect an account from your dashboard, or call `attach_account` with the
  one you want.
- **Attaching only adds.** An account already on a squid is never replaced,
  and automatic selection never runs on a squid that already has one.
- A rate-limited account is reported as temporarily locked, not missing —
  these locks are routine and clear on their own.

Call `attach_account` directly when you want to add, switch, or confirm an
account before running — for example, re-running an existing squid with a
different account than the one that was auto-picked. See
[Link Account to Squid](https://staging.docs.lobstr.io/docs/link-account-to-squid) for how account linking
works at the API level.

## A typical conversation

1. *"Find a scraper for Google Maps business leads."* → `search_scrapers`
2. *"What inputs does it need?"* → `get_scraper_details`
3. *"Run it for dentists in Manchester."* → `run_scraper` (asks you to confirm)
4. *"Is it done?"* → `get_run`
5. *"Show me the results."* → `get_results`

## Troubleshooting

**The client can't authorize / the connection fails.** The server must be
reached over HTTPS at `https://mcp.lobstr.io/mcp`. Make sure you copied the full
URL including `/mcp`.

**Every run asks for confirmation.** Expected — lobstr.io prices per row, so the
total is unknowable up front. The estimate still shows the per-row rate.

**A run stays pending or errors.** Check the run in your
[lobstr.io dashboard](https://app.lobstr.io); the MCP surfaces the same status
your account shows.
