What is XCrawl?
XCrawl is an API that turns websites into structured data for AI apps. One service covers scraping a page, crawling a site, mapping its URLs and fetching search results.
The site describes output as clean Markdown and JSON aimed at LLM and RAG workflows.
What can the API do?
It offers separate endpoints for scraping, searching, crawling and mapping, plus a SERP API and a ChatGPT API that the site marks as new.
- Scrape: a single URL into Markdown, JSON, HTML or a screenshot
- Crawl: a whole site, with depth and scope rules
- Map: every discoverable URL on a domain
- Search and SERP: structured results from Google, Bing and other engines
Does it handle JavaScript sites and proxies?
According to the site, yes. It lists headless browser rendering for JavaScript-heavy sites and single-page apps, and auto-rotating residential proxies with automatic retries.
The site also claims a 99%+ data extraction success rate. That is a marketing claim and not independently verified here.
How do I connect it to my stack?
Use the Python or Node.js SDK, the REST API, a CLI, or the MCP server for AI assistants. An n8n node is also available, according to the submission.
The site says results can be sent by webhook or delivered to S3 and GCS.
How much does XCrawl cost?
New accounts get 1,000 free credits with no credit card. Requests use credits based on complexity, and the site does not list plan prices on the pages checked.
The pricing page offers Scale plans for higher volume through Contact Sales. Check the pricing page for the current credit table.
Alternatives to XCrawl
Teams that need web data for AI apps either use a scraping API, a platform with ready-made scrapers, or build their own with proxies and headless browsers. XCrawl is a scraping API with crawl, map and search endpoints in one place.
- FirecrawlAn API that scrapes and crawls sites into Markdown or structured data for AI apps.firecrawl.dev ↗
- ApifyA platform for web scraping and automation with a marketplace of ready-made scrapers.apify.com ↗
- ScrapingBeeA scraping API that handles proxies and JavaScript rendering.scrapingbee.com ↗
- Jina ReaderA service that converts a URL into clean text for language models.jina.ai/reader ↗
How XCrawl compares
| Option | Best for | Trade-off |
|---|---|---|
| XCrawl | Developers who want scrape, crawl, map and search behind one API, with SDKs, an MCP server and n8n | Plan prices are not shown on the pages checked, so cost beyond the 1,000 free credits needs a look at the pricing page |
| Firecrawl | Teams that want a popular scrape and crawl API for LLM pipelines. | Credit based pricing can grow with crawl volume. |
| Apify | Ready-made scrapers for specific sites and custom automation. | A larger platform with more to learn than a single API. |
| ScrapingBee | Developers who want proxies and rendering handled behind one endpoint. | Returns page content, with fewer AI-specific output formats. |
| Jina Reader | Quickly turning single pages into LLM-ready text. | Focused on reading pages, not full site crawling. |
XCrawl suits developers who want one API for the main web data jobs and a quick start on 1,000 free credits. If you need prebuilt scrapers for particular sites or have a fixed budget in mind, compare Apify and Firecrawl and check each pricing page.
Questions people ask about XCrawl
What is XCrawl?
A web data API for developers that scrapes, crawls, maps and searches the web and returns Markdown, JSON, HTML or screenshots.
Is there a free tier?
New accounts start with 1,000 free credits and no credit card, according to the site.
Which languages have SDKs?
Python and Node.js. The site also shows REST examples for Go, Ruby, PHP and cURL.
Does it work with AI assistants?
The site lists an MCP server for Claude and other AI assistants.
Can it render JavaScript pages?
According to the site, it uses headless browser rendering for JavaScript-heavy sites, single-page apps and infinite scroll.
Does it return search results?
Yes. The Search and SERP APIs return structured results from Google, Bing and other engines, according to the submission.
How is usage billed?
In credits, charged by request complexity. The site points to a credit table on its pricing page.
What are the alternatives?
Firecrawl, ScrapingBee and Apify cover similar ground. See the comparison on this page.
