Pagination
How to iterate through large result sets using the lobstr.io API
All list endpoints return paginated responses. This guide explains the pagination model and how to fetch all records efficiently.
Response structure
Every paginated endpoint returns the same envelope:
{ "total_results": 243, "limit": 10, "page": 1, "total_pages": 25, "result_from": 1, "result_to": 10, "data": [...] }
| Field | Type | Description |
|---|---|---|
total_results | integer | Total number of matching records |
limit | integer | Number of records per page (max 100) |
page | integer | Current page number (1-indexed) |
total_pages | integer | Total number of pages |
result_from | integer | Index of the first record on this page |
result_to | integer | Index of the last record on this page |
data | array | The records for this page |
Parameters
| Parameter | Default | Max | Description |
|---|---|---|---|
page | 1 | — | Page number to retrieve |
page_size | 10 | 100 | Number of results per page |
Pass these as query parameters:
GET /v1/results?squid=abc123&page=2&page_size=50
The page-size query parameter's name varies by endpoint — Get Results uses page_size, while List Accounts uses limit. Check the specific endpoint's own page if you're unsure.
Iterating through all pages
import requests API_KEY = "YOUR_API_KEY" headers = {"Authorization": f"Token {API_KEY}"} def get_all_results(squid_id): all_data = [] page = 1 page_size = 100 while True: response = requests.get( "https://api.lobstr.io/v1/results", headers=headers, params={"squid": squid_id, "page": page, "page_size": page_size} ) body = response.json() all_data.extend(body["data"]) print(f"Fetched page {page}/{body['total_pages']} ({len(all_data)}/{body['total_results']} records)") if page >= body["total_pages"]: break page += 1 return all_data results = get_all_results("YOUR_SQUID_ID") print(f"Total records fetched: {len(results)}")
Free plan
Every plan, the free plan included, pages through every result: there is no row cap since November 3, 2026, and no warning field on the response.
Endpoints that support pagination
| Endpoint | Path |
|---|---|
| List Squids | GET /v1/squids |
| List Tasks | GET /v1/tasks |
| List Runs | GET /v1/runs |
| Get Run Tasks | GET /v1/runs/{run_hash}/tasks |
| Get Results | GET /v1/results |
| List Accounts | GET /v1/accounts |
Use the maximum page size (page_size=100 for Get Results; check each endpoint's own parameter name) to minimize the number of requests when fetching large datasets.