Skip to content
Examples

Add Tasks

Add Twitter search scraping tasks by providing search URLs

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/tasks

Add Twitter search results to scrape by providing search URLs from X.com (Twitter). The scraper will extract tweets matching your search query.

This scraper requires a synced Twitter/X account. Link one to your squid before running.
Crawler ID: 1b16ff414d27920fb325b68436dbf5fc

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Request body format. Value: application/json

Request Body

Field Type Required Description
squid string Yes Hash of your squid configured with the Twitter Search Results Scraper
tasks array Yes Array of task objects, each containing a url field

Task Object

Field Type Required Description
url string Yes Twitter search URL (e.g., https://x.com/search?q=keyword&src=typed_query&f=top)

Search URL Parameters

Twitter search URLs support various filters via query parameters:

Parameter Description
q Search query (URL encoded)
f=top Show top results
f=live Show latest results
f=user Show people results
f=image Show image results
f=video Show video results

Perform your search on Twitter/X first, apply filters, then copy the URL to preserve all search parameters.

Use f=live to get the most recent tweets matching your query.

Advanced search operators like from:username, since:2024-01-01, and filter:media are supported in the query.

Code Examples

curl -X POST "https://api.lobstr.io/v1/tasks" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "squid": "YOUR_SQUID_HASH",
    "tasks": [
      {
        "url": "https://x.com/search?q=artificial%20intelligence&src=typed_query&f=top"
      }
    ]
  }'

Response

200
{
  "duplicated_count": 0,
  "tasks": [
    {
      "id": "9fb224b5e2ce36942a74d1e94dd1cf14",
      "created_at": "2025-07-07T09:53:32.017225",
      "is_active": true,
      "params": {
        "url": "https://x.com/search?q=artificial%20intelligence&src=typed_query&f=top"
      },
      "module": 178,
      "object": "task"
    }
  ]
}