Skip to content
Examples

Add Tasks

Add keyword or topic tasks to scrape Google News articles

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/tasks

Add keyword or topic tasks to scrape Google News articles. Each task represents a search query or topic that will fetch matching articles from Google News.

Crawler ID: 06faf01896336c01ffb5e96d13dd406c

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Request body format. Value: application/json

Request Body

Field Type Required Description
squid string Yes Hash of your squid configured with the google-news-search-export crawler
tasks array Yes Array of task objects, each containing a keyword field

Task Object

Field Type Required Description
keyword string Yes A search query (e.g., climate change) or a predefined Google News topic in CAPS: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH

To scrape a specific topic page or section URL from news.google.com, set topic_url in the squid settings instead — task keywords are ignored when topic_url is set.

Configure time range, language, and date filters in the squid settings.

Code Examples

curl -X POST "https://api.lobstr.io/v1/tasks" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tasks": [
      {"keyword": "crab"},
      {"keyword": "climate change"}
    ],
    "squid": "YOUR_SQUID_HASH"
  }'

Response

200
{
  "duplicated_count": 0,
  "tasks": [
    {
      "id": "5f5ccd42b74cf18196dcb7fe14fb8030",
      "created_at": "2025-07-04T04:51:12.532884",
      "is_active": true,
      "params": {
        "keyword": "crab"
      },
      "module": 131,
      "object": "task"
    }
  ]
}