/agent by Firecrawl

Another standout capability is the multi-model parallel agent execution with an intelligent waterfall fallback system. Firecrawl offers multiple processing models: Spark 1 Fast for instant lookups, Spark 1 Mini for lightweight tasks at 60% lower cost, and Spark 1 Pro for high-complexity research. The platform orchestrates parallel agents that first attempt Spark 1 Fast for quick retrieval; if the query demands deeper investigation, it automatically upgrades to Spark 1 Mini or Pro without user intervention. This dynamic resource allocation ensures that simple requests, like looking up a company's founding team, return in seconds and cost minimal credits, while demanding research tasks—such as curating a dataset of all AI papers from arXiv—receive the necessary computational power. This flexibility makes the web agent API cost-effective and scalable, allowing teams to balance speed, accuracy, and budget seamlessly across different data collection campaigns.

Firecrawl /agent integrates deeply with the broader Firecrawl platform and developer ecosystems. It is accessible through a REST API, native SDKs for Python and Node.js, a command-line interface, and Model Context Protocols (MCPs), making it easy to embed into any application or workflow. Beyond the core agent, users can leverage companion Firecrawl services like /extract (now with optional URLs), /crawl for large-scale site traversal, /scrape for single-page data extraction, and /interact for simulated browser actions. These integrations allow you to chain agent-driven searches with dedicated scraping steps, handle multi-site data aggregation, and even navigate interfaces that require clicks or form submissions. The code examples on the site show how a few lines of Python or a simple cURL command can kick off an agent task, monitor progress, and retrieve the final structured result, which can then be piped directly into machine learning pipelines, analytics dashboards, or operational databases.

The overarching workflow of Firecrawl /agent is elegantly simple: from prompt to processed data. After submitting a prompt, the agent begins by parsing the request to understand the entities and attributes you’re after. It then identifies candidate sources—sometimes the Firecrawl Research Index, a specialized index for AI/ML research with state-of-the-art recall, or live web searches—and navigates to each candidate, performing the necessary interactions (scrolling, clicking, paging). The agent monitors its progress in real-time, indicating which sources are being extracted and the status of each. When complete, it compiles all records into a structured JSON output, often with hundreds of records, as demonstrated in the live example that extracted 247 records for a simple founder query. This entire process is transparent through the API, allowing users to log activity, set max credits to control spending, and retry as needed. By abstracting away the complexities of web automation, the web agent API lets you treat the web like a queryable database.

Real-world use cases for Firecrawl /agent span multiple industries. A venture capital analyst could run a lead generation task to “Get all YC W24 companies with founders,” instantly building a pipeline of potential investments with structured data on company name, founders, and website, all sourced from Y Combinator’s directory and crunchbase. An e-commerce competitor researcher might prompt “Get all Nike Air Jordan listings with prices” from nike.com, pulling product names, sizes, and current prices for market benchmarking. A financial data provider could request “Get market cap for top 50 tech stocks” to compile a daily-updated database. Academics and ML engineers can “Build a dataset of all AI papers from arXiv” to fuel training data or meta-research. Real estate platforms can track “3BR apartments in SF under $4k,” while food researchers extract “Michelin star restaurants in NYC.” In every case, the agent turns a vague data desire into a clean, ready-to-use dataset, often within minutes.

Firecrawl /agent is purpose-built for forward-thinking technical teams: software developers integrating data into AI agents or knowledge bases, data engineers building automated ETL pipelines, market researchers needing up-to-date competitive intelligence, and startup founders scouting market landscapes. The platform’s generous free tier includes 5 free daily runs, allowing anyone to experiment and prototype without a credit card. Paid usage follows a dynamic pricing model during the research preview, where simple queries cost fewer credits and complex ones consume more; you can set a `maxCredits` parameter to cap spending. Supported platforms include any environment that can make HTTP requests, with official clients for Python and Node.js, plus a CLI for scripting. In essence, Firecrawl /agent removes the last mile of web data extraction by providing a conversational, autonomous data retrieval system that works across any website—finally making the web a truly queryable resource for every developer.