Skip to main content
POST
/
api
/
products
/
queue-crawl
Queue Site Crawl
curl --request POST \
  --url https://api.merchantops.ai/api/products/queue-crawl \
  --header 'Content-Type: application/json' \
  --data '
{
  "site_url": "<string>",
  "max_pages": 500,
  "search": "product"
}
'
{
  "job_id": "<string>",
  "message": "<string>",
  "site_url": "<string>",
  "status": "<string>"
}

Headers

authorization
string | null
X-Organization-ID
string | null

Cookies

stytch_session
string | null

Body

application/json

Request model for site crawl job using Firecrawl Map API.

site_url
string
required

Base URL to map

max_pages
integer
default:500

Maximum URLs to discover (1-5000). Map API is efficient for large sites.

Required range: 1 <= x <= 5000

Search term to filter discovered URLs (e.g., 'product' to find product pages)

Response

Successful Response

Response model for crawl job.

job_id
string
required

Unique job identifier

message
string
required

Status message

site_url
string
required

Site being crawled

status
string
required

Job status