Skip to content
Examples

Update Settings

Configure the Reddit Scraper settings and filters

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
POSThttps://api.lobstr.io/v1/squids/{squid_hash}

Configure your Reddit Scraper squid settings. Control sort order per input type, filter by date, limit results, and choose whether to include posts, comments, or both.

Headers

Authorizationstringheaderrequired

Your API authentication token. Value: Token YOUR_API_KEY

Content-Typestringheaderrequired

Request body format. Value: application/json

Scraper Parameters

Set these parameters in the params object:

Parameter Type Default Description
max_results integer unlimited Maximum number of rows (posts + comments) to collect per task
max_unique_results_per_run integer unlimited Maximum unique results across all tasks in the run
fetch_since string null Stop collecting content older than this threshold. The task ends at the first older post, so use it with sort_subreddit set to new. Use a relative duration (24h, 7d, 2w) or absolute date (YYYY-MM-DD HH:MM:SS)
fetch_since_timezone string null Timezone for interpreting an absolute fetch_since date (e.g. Europe/Paris). Ignored for relative values
sort_comments string best Sort order for post thread comments: best, top, new, controversial, old, q&a
sort_search string relevance Sort order for search results: relevance, hot, top, new, comments. With relevance, top or comments, the time window is set by sort_period
sort_subreddit string hot Sort order for subreddit feeds: hot, new, top, rising, controversial. With top and controversial, the time window is set by sort_period
sort_period string all Time window of the ranking: all, year, month, week, day, hour. Applies to subreddit feeds sorted by top or controversial, and to searches sorted by relevance, top or comments. A t already in the task URL (e.g. https://www.reddit.com/r/Bitcoin/top/?t=year) wins
include_nsfw boolean false Include NSFW (adult) posts and comments in results
skip_comments boolean false Return only post rows — skip all comments
skip_posts boolean false Skip post rows. For post URLs: returns comments only. For user URLs: returns profile and comments. For subreddit URLs: returns subreddit metadata only
max_depth integer unlimited Maximum nesting depth of comments to collect. 0 = top-level comments only, 1 = top-level + first replies, etc. Leave empty for full thread
keyword string null Search keyword. Automatically constructs a Reddit search URL for the given keyword and subreddit(s) — no need to manually craft search URLs
post_type string any Filter search results by post type: any, image, video. Wired to Reddit's real media tabs; other values (previously link, self, richvideo) are not accepted by Reddit's search and have been removed. Only applies to search URLs and keyword searches
search_comments boolean false Search within comment bodies instead of post titles. Only applies to search URLs and keyword searches

Squid Settings

Setting Type Description
name string Display name for your squid configuration
concurrency integer Number of parallel scraping threads (default: 1)
export_unique_results boolean Export only unique results (deduplicated)
to_complete boolean Run until all tasks complete
no_line_breaks boolean Remove line breaks from results

The sort_comments, sort_search, and sort_subreddit parameters are each applied only when the input URL matches the corresponding type — they do not conflict with each other.

fetch_since filters based on the created_at field of each post or comment. It is useful for incremental runs to avoid re-collecting old content.

Reddit serves about 1,000 posts per feed at most (new, top, controversial, search). On a busy subreddit, new only reaches back a few weeks. To go back further, add the same subreddit several times with different sorts and periods in the URL, plus keyword searches scoped to it, then dedupe on post_id.

Code Examples

curl -X POST "https://api.lobstr.io/v1/squids/YOUR_SQUID_HASH" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Reddit Scraper (1)",
    "concurrency": 1,
    "export_unique_results": true,
    "to_complete": false,
    "no_line_breaks": true,
    "params": {
      "max_results": 500,
      "fetch_since": "7d",
      "sort_subreddit": "hot",
      "sort_comments": "top",
      "skip_comments": false,
      "skip_posts": false,
      "include_nsfw": false
    }
  }'

Response

201
{
  "name": "Reddit Scraper (1)",
  "concurrency": 1,
  "export_unique_results": true,
  "to_complete": false,
  "no_line_breaks": true,
  "params": {
    "max_results": 500,
    "fetch_since": "7d",
    "sort_subreddit": "hot",
    "sort_comments": "top",
    "skip_comments": false,
    "skip_posts": false,
    "include_nsfw": false
  }
}