Skip to content
Examples

Update Settings

Configure the Google Search Scraper parameters

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/squids/{squid_hash}

Configure the scraper parameters to control how Google Search results are collected. These settings include pagination, location and language.

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Request body format. Value: application/json

Scraper Parameters

Set these parameters in the params object:

Parameter Type Description
max_pages integer Maximum search result pages to crawl per keyword. Google limits to ~300-400 total results. The scraper always requests 10 results per page and paginates.
max_unique_results_per_run integer Maximum number of unique results retrieved across all tasks in the entire run. Leave empty for unlimited.
country string Sets IP origin country and Google domain (e.g., google.fr for France). Default: United States
location string Geographic bias for results (state, city). Use Get Geolocation to find valid values.
language string Language for search results. Defaults to English for the chosen country.
collect_shopping_results boolean Also collect the Google Shopping results for each keyword (first page, up to 40 products): seller, price, rating, reviews and image. Uses the same country, language and location as the search. Billed at 1 credit per shopping result. Default: false

Use the Get Geolocation endpoint to discover valid location values.

Higher max_pages values will collect more results but use more credits.

Code Examples

curl -X POST "https://api.lobstr.io/v1/squids/YOUR_SQUID_HASH" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Google Search Scraper",
    "concurrency": 1,
    "export_unique_results": true,
    "to_complete": false,
    "no_line_breaks": true,
    "params": {
      "country": "United States",
      "language": "English",
      "max_pages": 4,
      "max_unique_results_per_run": 500,
      "location": "Alabama",
      "collect_shopping_results": false
    }
  }'

Response

201
{
  "name": "Google Search Scraper",
  "no_line_breaks": true,
  "params": {
    "country": "United States",
    "language": "English",
    "max_pages": 4,
    "max_unique_results_per_run": 500,
    "location": "Alabama",
    "collect_shopping_results": false
  },
  "to_complete": false,
  "export_unique_results": true
}