Skip to content
Developer Tools

Python SDK

Official Python SDK for the lobstr.io API. Sync and async clients, typed models, auto-pagination, and full API coverage.

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.

Installation

pip install lobstrio-sdk
Sync & Async

Both LobstrClient and AsyncLobstrClient with identical API surfaces.

Typed Models

Dataclass models for every response — no raw dicts in the public API.

Auto-Pagination

Lazy PageIterator streams all pages on demand with .iter() method.

Automatic Auth

Token resolved from explicit param, LOBSTR_TOKEN env, or ~/.config/lobstr/config.toml.

Authentication

The SDK resolves your API token in this order:

  1. Explicit parameter — LobstrClient(token="your_token")
  2. Environment variable — LOBSTR_TOKEN=your_token
  3. CLI config file — ~/.config/lobstr/config.toml (written by the lobstr CLI on login)

If LOBSTR_TOKEN is not set, the SDK silently falls back to the CLI config file — which may belong to a different account. There is no warning when the env var is absent. Always set LOBSTR_TOKEN explicitly in production environments.

export LOBSTR_TOKEN=your_api_key

Quick Start

import time
from lobstrio import LobstrClient

# Token resolved from LOBSTR_TOKEN env var, then ~/.config/lobstr/config.toml.
# Set LOBSTR_TOKEN explicitly in production — misnamed env vars fail silently.
client = LobstrClient()

# 1. Resolve crawler slug → id (create() requires the hash, not the slug)
crawlers = client.crawlers.list()
crawler = next(c for c in crawlers if c.slug == "google-maps-leads-scraper")

# 2. Create a squid
squid = client.squids.create(crawler=crawler.id, name="Restaurants Paris")

# 3. Configure squid-level params — required before runs.start(), even when all
#    params are optional. Pass None for params you don't need to set.
client.squids.update(squid.id, params={"language": "English (United States)", "max_results": 50})

# 4. Add tasks
client.tasks.add(squid=squid.id, tasks=[
    {"url": "https://www.google.com/maps/search/restaurants/@48.8566,2.3522,14z"}
])

# 5. Start run and stream progress
def on_progress(stats):
    print(f"  {stats.percent_done} — {stats.total_results} results, ETA {stats.eta}")

run = client.runs.start(squid=squid.id)
print(f"Run {run.id} started")
run = client.runs.wait(run.id, callback=on_progress)
# Shortcut: client.runs.call(squid=squid.id, callback=on_progress) does the
# start + wait in one call. Add timeout=<seconds> to either to bound the wait.
print(f"Run complete — {run.total_results} results, {run.credit_used} credits used")

# 6. Wait for export to be ready before fetching results
while not run.export_done:
    time.sleep(3)
    run = client.runs.get(run.id)

# 7. Iterate results (PageIterator yields one dict per result)
for place in client.results.iter(squid=squid.id):
    print(f"{place['name']} — {place.get('score', 'N/A')}★  {place.get('phone', '')}")

# 8. Clean up — delete the squid you no longer need
client.squids.delete(squid.id)

API Reference

User

SDK Method HTTP Endpoint Description
client.me() GET /v1/me Get authenticated user profile
client.balance() GET /v1/user/balance Get current credit balance

Balance.available is your plan's total allotment, not credits remaining. Remaining credits = available − consumed. Example: Balance(available=100, consumed=64) means 36 credits left.

Crawlers

SDK Method HTTP Endpoint Description
client.crawlers.list() GET /v1/crawlers List all available crawlers
client.crawlers.get(hash) GET /v1/crawlers/{hash} Get crawler details by hash
client.crawlers.params(hash) GET /v1/crawlers/{hash}/params Get supported parameters for a crawler
client.crawlers.attributes(hash) GET /v1/crawlers/{hash}/attributes Get output attributes for a crawler

A Crawler has both account (bool — needs an account or not) and account_type (the account type slug it needs, e.g. linkedin-sync, sales-nav-sync, or None if it needs no account). Two crawlers for the same platform can need different account types, so match a connected account on account_type, not on the crawler's name.

Squids

SDK Method HTTP Endpoint Description
client.squids.create(crawler, name) POST /v1/squids Create a new squid for a crawler
client.squids.list() GET /v1/squids List all squids in your account
client.squids.get(hash) GET /v1/squids/{hash} Get squid details by hash
client.squids.update(hash, ..., accounts=[...]) POST /v1/squids/{hash} Update squid settings, parameters, or the linked account list
client.squids.attach_accounts(hash, accounts, replace=False) GET + POST /v1/squids/{hash} Link one or more connected accounts to a squid without dropping the ones already there
client.squids.empty(hash) POST /v1/squids/{hash}/empty Remove all tasks from a squid
client.squids.delete(hash) DELETE /v1/squids/{hash} Permanently delete a squid

Linking an account. accounts on update() is a direct mirror of the API field, which is full-replace — sending it detaches any account not included, and leaving it unset (the default) leaves the squid's accounts untouched. attach_accounts() is the safe way to add one: it reads the squid's current accounts first and sends the union, so nothing already attached is dropped. Pass replace=True to set the list outright instead, including [] to detach everything. Both are available on LobstrClient and AsyncLobstrClient. The read and the write inside attach_accounts() are two separate requests, not atomic — a concurrent attach on the same squid between them can still drop one addition.

Tasks

SDK Method HTTP Endpoint Description
client.tasks.add(squid, tasks) POST /v1/tasks Add one or more tasks to a squid
client.tasks.list(squid) GET /v1/tasks List tasks for a squid
client.tasks.get(hash) GET /v1/tasks/{hash} Get a single task by hash
client.tasks.upload(squid, file) POST /v1/tasks/upload Bulk upload tasks from a file
client.tasks.upload_status(id) GET /v1/tasks/upload/{id} Check status of a bulk upload
client.tasks.delete(hash) DELETE /v1/tasks/{hash} Delete a task by hash

Runs

SDK Method HTTP Endpoint Description
client.runs.start(squid) POST /v1/runs Start a new run and return the Run right away, without waiting
client.runs.call(squid, timeout=None, callback=None) POST + poll /v1/runs Start a run and wait for it to finish — returns the finished run. Combines start() + wait()
client.runs.list(squid) GET /v1/runs List all runs for a squid
client.runs.get(hash) GET /v1/runs/{hash} Get run details by hash
client.runs.stats(hash) GET /v1/runs/{hash}/stats Get statistics for a run
client.runs.tasks(hash) GET /v1/runtasks List tasks executed in a run
client.runs.abort(hash) POST /v1/runs/{hash}/abort Abort an in-progress run
client.runs.download_url(hash) GET /v1/runs/{hash}/download Get S3 download URL for run results
client.runs.download(hash, dest) GET /v1/runs/{hash}/download Download run results to a local file
client.runs.wait(hash, timeout=None, callback=None) — — Poll until the run completes; optional callback(RunStats) for progress. Pass timeout (seconds) to bound the wait

start() vs call(). start() launches a run and hands back the Run immediately — you poll its progress or come back to it later. call() launches a run and waits until it finishes, returning the finished Run (the same as start() followed by wait()). Both accept timeout on the waiting side.

Timeouts. wait() and call() wait indefinitely by default (timeout=None). Set timeout to a number of seconds to give up waiting — a RunTimeout (a subclass of the built-in TimeoutError) is raised. The run is not aborted; it keeps running server-side, so you can re-attach later with runs.wait(run_id) or runs.get(run_id).

from lobstrio import RunTimeout

try:
    run = client.runs.call(squid=squid.id, timeout=300)  # give up after 5 min
except RunTimeout as e:
    print(f"Still running — re-attach later with runs.wait('{e.run_id}')")

Results

SDK Method HTTP Endpoint Description
client.results.list(squid) GET /v1/results Get first page of results for a squid
client.results.iter(squid) GET /v1/results Lazily iterate all result pages for a squid

Accounts

SDK Method HTTP Endpoint Description
client.accounts.list() GET /v1/accounts List all connected accounts
client.accounts.get(hash) GET /v1/accounts/{hash} Get account details by hash
client.accounts.types() GET /v1/accounts/types List supported account types
client.accounts.sync(type, cookies) POST /v1/accounts/cookies Sync an account using cookies
client.accounts.delete(hash) DELETE /v1/accounts/{hash} Delete a connected account

An Account also carries status, resets_in, and lock_time — what a client needs to tell whether the account is healthy and available, or currently locked by a run on another squid, before picking it to attach. There is no attach call on AccountsResource itself; link an account to a squid with client.squids.attach_accounts() above.

Delivery

SDK Method HTTP Endpoint Description
client.delivery.email(squid, ...) POST /v1/delivery Configure email delivery for a squid
client.delivery.google_sheet(squid, ...) POST /v1/delivery Configure Google Sheets delivery
client.delivery.s3(squid, ...) POST /v1/delivery Configure Amazon S3 delivery
client.delivery.webhook(squid, ...) POST /v1/delivery Configure webhook delivery
client.delivery.sftp(squid, ...) POST /v1/delivery Configure SFTP delivery