Skip to content
Examples

Update Settings

Configure Twitter Profile Scraper settings including raw JSON output

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/squids/{squid_hash}

Configure your Twitter Profile Scraper squid settings. Control output format, concurrency, and whether to return raw JSON data from Twitter's API.

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Request body format. Value: application/json

Scraper Parameters

Set these parameters in the params object:

Parameter Type Default Description
json boolean false If true, returns the complete raw JSON payload for the user profile

Squid Settings

Configure general squid settings:

Setting Type Description
name string Display name for your squid configuration
concurrency integer Number of parallel scraping threads (default: 1)
export_unique_results boolean Export only unique results (deduplicated)
to_complete boolean Run until all tasks complete
no_line_breaks boolean Remove line breaks from results

Enable json: true to get the complete raw API response for advanced analysis or custom field extraction.

The raw JSON contains additional fields not included in the parsed output.

Code Examples

curl -X POST "https://api.lobstr.io/v1/squids/YOUR_SQUID_HASH" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Twitter Profile Scraper (2)",
    "concurrency": 1,
    "export_unique_results": true,
    "to_complete": false,
    "no_line_breaks": true,
    "params": {
      "json": true
    }
  }'

Response

201
{
  "name": "Twitter Profile Scraper (2)",
  "no_line_breaks": true,
  "params": {
    "json": true
  },
  "to_complete": false,
  "export_unique_results": true
}