Get Crawler Parameters
Retrieve the list of configurable input parameters for a specific crawler
https://api.lobstr.io/v1/crawlers/{crawler_hash}/paramsThis endpoint returns all configurable input parameters for a specific crawler. The response is organized into two main sections: task and squid.
Response Structure
The response contains:
- task: Object containing task-level parameters (parameter name as key)
- squid: Object containing squid-level parameters and functions
- squid.functions: Optional add-on features with extra credit costs
Task-level Parameter Fields
task.{parameter_name}.typestringData type (string, int, boolean, etc.). Example: "string"
task.{parameter_name}.groupstringLogical grouping (e.g., "url", "location"). Example: "url"
task.{parameter_name}.regexstringValidation pattern for string inputs. Example: "^https?://.*"
task.{parameter_name}.requiredbooleanWhether the parameter is mandatory. Example: true
task.{parameter_name}.defaultstring | number | boolean | nullDefault value if not provided. Example: null
task.{parameter_name}.attributestringInternal attribute name. Example: "url"
task.{parameter_name}.is_paramsbooleanWhether it's used for URL parameter generation. Example: true
Squid-level Parameter Fields
squid.{parameter_name}.typestringData type of the parameter. Example: "string"
squid.{parameter_name}.allowedarrayArray of valid values (for enum types). Example: ["en", "es", "fr"]
squid.{parameter_name}.defaultstring | number | boolean | nullDefault value if not provided. Example: "en"
squid.{parameter_name}.requiredbooleanWhether the parameter is mandatory. Example: false
squid.{parameter_name}.attributestringInternal attribute name. Example: "language"
Functions (Optional Add-ons) Fields
squid.functions.{function_name}.sortintegerDisplay order of the function. Example: 1
squid.functions.{function_name}.defaultbooleanWhether the function is enabled by default. Example: false
squid.functions.{function_name}.credits_per_functionobjectExtra credits per result when this function is on, keyed by plan type like credits_per_row: current is the rate on today's plans, legacy the rate kept by grandfathered subscriptions (Get User Profile → plan[].is_legacy). Example: {"legacy": 80, "current": 4}
squid.functions.{function_name}.skip_squidbooleanIf true, a result this squid already ran the function on is not processed again, and not charged again, on later runs. Example: true
Headers
AuthorizationstringheaderrequiredYour API authentication token. Value: Token YOUR_API_KEY
Content-TypestringheaderDefault application/jsonContent type of the request. Value: application/json
Path Parameters
crawler_hashstringpathrequiredThe unique identifier (hash/id) of the crawler. Example: 4734d096159ef05210e0e1677e8be823
The response structure uses parameter names as object keys. Access task parameters via params.task.url, squid parameters via params.squid.language, etc.
Functions under squid.functions are optional add-ons. Each function adds its credits_per_function cost (the current or legacy rate) to every result it runs on.
Pay attention to 'allowed' arrays in squid parameters - these only accept specific values. Sending invalid values will cause validation errors.
Use 'regex' patterns in task parameters to validate input before sending. This helps catch errors early and reduces failed tasks.
Parameters with 'group' field are logically related. For location-based crawlers, you'll typically need to provide all parameters in the 'location' group.
Code Examples
curl -X GET "https://api.lobstr.io/v1/crawlers/4734d096159ef05210e0e1677e8be823/params" \ -H "Authorization: Token YOUR_API_KEY"
Response
{ "task": { "url": { "type": "string", "group": "url", "regex": "http(.*)google(.*)/maps/(search|place)(.*)", "default": null, "required": true, "attribute": "url", "is_params": false }, "city": { "type": "string", "group": "location", "default": null, "required": true, "attribute": "city", "is_params": true }, "region": { "type": "string", "group": "location", "default": null, "required": false, "attribute": "region", "is_params": true }, "country": { "type": "string", "group": "location", "default": null, "required": true, "attribute": "country", "is_params": true }, "category": { "type": "string", "group": "location", "default": null, "required": true, "attribute": "category", "is_params": true }, "district": { "type": "string", "group": "location", "default": null, "required": false, "attribute": "district", "is_params": true } }, "squid": { "country": { "type": "string", "allowed": [ "Afghanistan", "Albania", "Algeria", "United States", "United Kingdom", "France", "Germany", "Japan", "Brazil" ], "default": "United States", "required": false, "attribute": "country", "is_params": false }, "ratings": { "type": "string", "allowed": [ "Any rating", "2.0+", "2.5+", "3.0+", "3.5+", "4.0+", "4.5+" ], "default": "Any rating", "required": false, "attribute": "ratings", "is_params": false }, "language": { "type": "string", "allowed": [ "Afrikaans", "English (United States)", "Español (España)", "Français (France)", "Deutsch (Deutschland)", "Italiano", "Português (Brasil)", "日本語", "简体中文", "繁體中文" ], "default": "English (United States)", "required": true, "attribute": "language", "is_params": false }, "functions": { "fetch_business_images": { "sort": 1, "default": false, "skip_squid": true, "credits_per_function": { "legacy": 10, "current": 1 } }, "collect_business_details": { "sort": 0, "default": false, "skip_squid": true, "credits_per_function": { "legacy": 10, "current": 1 } }, "extract_emails_from_website": { "sort": 2, "default": false, "skip_squid": true, "credits_per_function": { "legacy": 10, "current": 1 } } } } }