Skip to content
Getting Started

Rate Limiting

Understanding API rate limits and how to handle them

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.

The lobstr.io API enforces rate limits to ensure fair usage and stable service performance. When you exceed the rate limit, the API returns a 429 Too Many Requests response.

Rate Limit Headers

Every API response includes headers to help you track your rate limit usage:

Header Description
X-RateLimit-Limit Total requests allowed per minute
X-RateLimit-Remaining Number of requests remaining in the current rate limit window
X-RateLimit-Reset UNIX timestamp indicating when the rate limit window resets
Retry-After Seconds until you can retry after hitting the limit

Endpoint-Specific Limits

Different endpoints have different rate limits based on their resource intensity:

Endpoint Limit
/v1/squids 120 requests/minute
/v1/tasks 90 requests/minute
/v1/runtasks 90 requests/minute
/v1/runs 120 requests/minute
/v1/results 2 requests/second

Rate Limit Error Response

When you exceed the rate limit, you'll receive a 429 response:

{
  "error": "Rate limit exceeded. Please try again later.",
  "type": "RateLimitExceeded",
  "code": 429
}

Example Response Headers

X-RateLimit-Limit: 120
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1716909300
Retry-After: 45

Monitor the X-RateLimit-Remaining header to proactively slow down requests before hitting the limit.

The /v1/results endpoint has a stricter limit (2 req/sec) due to the data-intensive nature of result retrieval.

Repeatedly hitting rate limits may result in temporary IP blocks. Implement exponential backoff in your code.

Use the Retry-After header value to know exactly when to retry your request.

Code Examples

curl -i -X GET "https://api.lobstr.io/v1/results" \
  -H "Authorization: Token YOUR_API_KEY"

Response

429
{
  "error": "Rate limit exceeded. Please try again later.",
  "type": "RateLimitExceeded",
  "code": 429
}