Skip to content
Examples

Update Settings

Configure the Google News Search Export parameters

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/squids/{squid_hash}

Configure the scraper parameters to control how Google News articles are collected. These settings include language, recency filters, result limits, and topic-based scraping.

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Request body format. Value: application/json

Scraper Parameters

Set these parameters in the params object:

Parameter Type Default Description
language string — Country edition and UI language for Google News (e.g., English (United States))
time_range string — Filter by recency: Any time, Past hour, Past 24h, Past week, Past year. The tighter of time_range and fetch_since wins.
max_results integer 100 Maximum articles to scrape per task. For keyword tasks requesting more than 100, the scraper walks backwards day by day from today (~100 articles per day). For topic tasks, limited to what the topic page returns (~100–200 articles).
topic_url string — Scrape a Google News topic instead of task keywords. Accepts a predefined topic name (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH) or a topic/section URL from news.google.com (https://news.google.com/topics/{id} or .../sections/{sid}). When set, task keywords are ignored — a single task is enough.
fetch_since string — Stop collecting articles older than this threshold. Use a relative duration (24h, 7d, 2w) or an absolute date (YYYY-MM-DD HH:MM:SS). Combines with time_range — the tighter constraint wins.
fetch_since_timezone string — Timezone for interpreting an absolute fetch_since date (e.g. Europe/Paris). Ignored for relative values.

Use topic_url: TECHNOLOGY to monitor a full Google News topic section without managing individual keyword tasks.

Use fetch_since: 7d combined with time_range: Past week to ensure you only collect recent articles.

Code Examples

curl -X POST "https://api.lobstr.io/v1/squids/YOUR_SQUID_HASH" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Google News Search Export",
    "concurrency": 1,
    "export_unique_results": true,
    "to_complete": false,
    "no_line_breaks": true,
    "params": {
      "language": "English (United States)",
      "time_range": "Past week",
      "max_results": 100,
      "fetch_since": "7d"
    }
  }'

Response

201
{
  "name": "Google News Search Export",
  "no_line_breaks": true,
  "params": {
    "language": "English (United States)",
    "time_range": "Past week",
    "max_results": 100,
    "fetch_since": "7d"
  },
  "to_complete": false,
  "export_unique_results": true
}