XCrawl exposes scrape, crawl, search and site-map APIs that return JSON, Markdown or screenshots. Python and Node SDKs, a CLI and an MCP integration let agents retrieve current web pages for research and question answering. The service handles proxies and JavaScript rendering; users must still respect each source site’s access rules.
Features
- Scrape, crawl, search and map APIs
- Python, Node, CLI and MCP access
Use Cases
- Feed traceable web material into a question-answering agent
FAQ
XCrawl exposes scrape, crawl, search and site-map APIs that return JSON, Markdown or screenshots. Python and Node SDKs, a CLI and an MCP integration let agents retrieve current web pages for research and question answering. The service handles proxies and JavaScript rendering; users must still respect each source site’s access rules. Core capabilities include: Scrape, crawl, search and map APIs, Python, Node, CLI and MCP access.
Common scenarios include: Feed traceable web material into a question-answering agent.
