Skip to content
Concepts

Pagination

A call is one page. When there is more, the answer holds the whole call for the next one.

A search or a list answers one page of the site, as the site shows it: 20 places on Google Maps, 35 listings on Leboncoin. When there is more, has_more is true and next is the next call, whole. Send it as it is; it is null after the last page.

{
  "has_more": true,
  "next": {
    "source": "google_maps",
    "operation": "reviews/list",
    "options": {
      "place": "ChIJc_459AHr9EcRd0ALc7vrjcI",
      "sort": "newest",
      "language": "en",
      "cursor": "gAAAAABq_G0yXBz0j4bINFws…"
    }
  }
}

Every page, in a loop

import os, requests

call = {
    "source": "google_maps",
    "operation": "reviews/list",
    "options": {
        "place": "ChIJc_459AHr9EcRd0ALc7vrjcI",
    },
}
reviews = []
while call and len(reviews) < 25:
    r = requests.post(
        "https://api.lobstr.io/v2/run",
        headers={"Authorization": f"Bearer {os.environ['LOBSTR_API_KEY']}"},
        json=call,
    )
    r.raise_for_status()
    answer = r.json()
    reviews += answer["results"]
    call = answer["next"]

Stop when you have enough: each page is a call, at the operation's price. A page without records costs nothing.

The cursor

cursor is opaque: it holds where the site's own paging stands, sealed. It only goes on with the search it came from: changed options answer invalid_cursor. Leave it out to start again at the first page.

With limit

limit trims the answer, not the page. With "limit": 2, a page of 10 reviews answers reviews 1 and 2, and next goes on at review 11.

Operations without a next page

A get or a check answers one record. Some searches answer everything in one call, as Google News' 100 articles: their pages say so, and next is always null.