Skip to content
Examples

Add Tasks

Add tasks to your Similarweb Scraper squid

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/tasks

Add domains or Similarweb URLs to your Similarweb Scraper squid.

Crawler ID: ac02527b552b088c1f4ebe5a41d3d40a

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Must be application/json. Value: application/json

Body

squidstringrequired

Hash ID of your squid.

tasksarrayrequired

Array of task objects.

Task Fields

tasks[].urlstringrequired

What to analyse. Four forms are accepted:

  • a bare domain: github.com
  • the website itself: https://github.com/
  • its Similarweb page: https://www.similarweb.com/website/github.com/
  • a Similarweb ranking: https://www.similarweb.com/top-websites/france/news-and-media/ — every website of the ranking is collected, one row each.

A ranking URL already carries its country and category: when it does, the country and category filters are ignored for that task.

Code Examples

curl -X POST https://api.lobstr.io/v1/tasks \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "squid": "YOUR_SQUID_HASH",
    "tasks": [
      {"url": "github.com"}
    ]
  }'

Response

200
{"duplicated_count": 0, "tasks": [{"id": "TASK_HASH", "is_active": true, "params": {"url": "github.com"}, "object": "task"}]}