Skip to content
Crawler

Get Crawler Attributes

Retrieve the list of output attributes (result fields) for a specific crawler

The v1 API, for batchesSquids, tasks and runs: many calls on a schedule, delivered where you work. For one call that answers in seconds, use the API v2.
GEThttps://api.lobstr.io/v1/crawlers/{crawler_hash}/attributes

This endpoint returns all output attributes (result fields) for a specific crawler. Each attribute describes a field that may appear in your scraping results, along with its data type, an example value, and the function it belongs to.

This is useful for understanding the shape of data a crawler produces before you start scraping, or for building dynamic result parsers.

Attribute Fields

namestring

The field name as it appears in result objects. Example: "name"

typestring

Data type of the field (string, integer, float, boolean, text, json). Example: "string"

exampleany

Example value for the field. May be null for optional fields. Example: "Blues Billard Club"

functionstring

The crawler function this attribute belongs to. Example: "Export Local Businesses"

is_mainboolean

Whether this attribute is included by default (true) or requires enabling a function (false). Example: true

descriptionstring

Human-readable description of what this field contains. Example: "Business name as listed on Google Maps"

Main vs Function Attributes

Attributes are divided into two categories:

  • Main attributes (is_main: true): Always included in results by default.
  • Function attributes (is_main: false): Only included when their parent function is enabled on your squid (e.g., "Collect Business Details", "Extract Emails from Website").

Headers

Content-TypestringheaderDefault application/json

Content type of the request. Value: application/json

Query Parameters

crawler_hashstringqueryrequired

The unique identifier (hash/id) of the crawler. Example: 4734d096159ef05210e0e1677e8be823

This is a public endpoint — no authentication is required.

Attributes with is_main: false only appear in results when their parent function is enabled on your squid. Check the function field to see which add-on controls each attribute.

Use the type field to build typed parsers for your results. Supported types are: string, integer, float, boolean, text (long string), and json (structured data).

Combine this endpoint with Get Crawler Parameters to fully understand both the inputs and outputs of any crawler before integrating it.

Code Examples

curl -X GET "https://api.lobstr.io/v1/crawlers/4734d096159ef05210e0e1677e8be823/attributes"

Response

200
{
  "data": [
    {
      "name": "match_filters",
      "type": "boolean",
      "example": false,
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Whether the result passed your geo and category filters"
    },
    {
      "name": "name",
      "type": "string",
      "example": "Blues Billard Club",
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Business name as listed on Google Maps"
    },
    {
      "name": "address",
      "type": "string",
      "example": "06110 Le Cannet, France",
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Full formatted address of the business"
    },
    {
      "name": "phone",
      "type": "string",
      "example": "+33493611118",
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Primary phone number of the business"
    },
    {
      "name": "website",
      "type": "string",
      "example": "http://www.example.com/",
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Business website URL"
    },
    {
      "name": "score",
      "type": "float",
      "example": 3.5,
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Average star rating out of 5"
    },
    {
      "name": "category",
      "type": "text",
      "example": "Fishing charter",
      "function": "Export Local Businesses",
      "is_main": true,
      "description": "Primary business category on Google Maps"
    },
    {
      "name": "email",
      "type": "string",
      "example": "info@lobstr.io",
      "function": "Extract Emails from Website",
      "is_main": false,
      "description": "Email address found on the business website"
    },
    {
      "name": "email_status",
      "type": "string",
      "example": "valid",
      "function": "Extract Emails from Website",
      "is_main": false,
      "description": "Validation status of the extracted email address"
    }
  ]
}