Skip to content
Squid

Configure Squid Chain

Chain two crawlers together so results from one automatically feed into another

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/squids/{id}/chain

Crawler chaining lets you connect two squids so that when a run on the source squid completes, its results are automatically extracted and queued as tasks on a target squid, triggering a new run downstream.

Example use case: Run a Google Maps Leads scrape to collect place URLs, then automatically pass those URLs into a Google Maps Reviews scrape — without any manual intervention.

Chains are limited to a maximum depth of 3 squids. A squid can only have one outgoing chain at a time.

How it works

  1. You configure a chain on a source squid, specifying the target crawler and a field_map
  2. When a run on the source squid completes with a status of DONE, the scheduler extracts the mapped fields from its results
  3. Those values are queued as tasks on the target squid (auto-created on first trigger if it doesn't exist yet)
  4. If autostart is true, a new run starts immediately on the target squid

The chain is only triggered when the source run reaches a status of DONE. Runs that end with ERROR or any other status will not trigger the downstream chain.

Path Parameters

idstringpathrequired

Hash ID of the source squid. Example: "e86b29c032024b66aff529e1d43c2bd7"

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Must be application/json. Value: application/json

Request Body

target_module_idstringrequired

Hash ID of the target crawler. Use GET /v1/crawlers to find it. Example: "9a3f1c8b2e7d4a56f0e1b2c3d4e5f678"

field_mapobjectrequired

Maps a result field from the source crawler to an input parameter on the target crawler. Example: {"url": "url"} — passes the url field from source results as the url input on the target.

autostartboolean

If true, the downstream run starts automatically when tasks are queued. If false, tasks are queued and the run is created as paused for manual review. Defaults to true.

cluster_namestring

Custom name for the auto-created target squid. If omitted, defaults to "{Crawler Name} (N) (chained)".

cluster_concurrencyinteger

Concurrency to apply to the auto-created target squid. Defaults to 1.

Response Fields

idstring

Unique chain config identifier.

target_moduleobject

Details of the target crawler.

Show target_module fields
idstring
Crawler hash ID
public_namestring
Human-readable crawler name
max_concurrencyinteger
Maximum concurrency supported by this crawler
target_clusterobject | null

The auto-created target squid. null until the first chain trigger fires.

field_mapobject

The configured field mapping.

is_activeboolean

Whether the chain is currently active.

autostartboolean

Whether downstream runs start automatically.

cluster_namestring | null

Configured name for the target squid.

cluster_concurrencyinteger | null

Configured concurrency for the target squid.

created_atstring

ISO 8601 timestamp of when the chain was configured.

Code Examples

curl -X POST "https://api.lobstr.io/v1/squids/e86b29c032024b66aff529e1d43c2bd7/chain" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "target_module_id": "9a3f1c8b2e7d4a56f0e1b2c3d4e5f678",
    "field_map": {"url": "url"},
    "autostart": true,
    "cluster_name": "Reviews (chained)",
    "cluster_concurrency": 3
  }'

Response

201
{
  "id": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4",
  "target_module": {
    "id": "9a3f1c8b2e7d4a56f0e1b2c3d4e5f678",
    "public_name": "Google Maps Reviews Scraper",
    "name": "googlemapreviews_matrix",
    "max_concurrency": 10
  },
  "target_cluster": null,
  "field_map": {"url": "url"},
  "is_active": true,
  "autostart": true,
  "cluster_name": "Reviews (chained)",
  "cluster_concurrency": 3,
  "created_at": "2026-04-28T10:00:00Z"
}
400
{
  "error": "field_map is required and must be a dict."
}
409
{
  "error": "This squid already has a chain configured."
}
422
{
  "error": "Chain depth limit of 3 reached."
}