better-crawler-4-agent

better-crawler-4-agent is an MCP server designed for web scraping challenges facing intelligent agents. It uses real browser rendering to address issues where HTTP scraping returns empty page shells or is blocked by anti-bot measures. The plugin provides three tools: single-page scraping, concurrent scraping (supporting 20 concurrent tasks), and status checking. It adopts a tiered fallback strategy that prioritizes browser rendering and automatically falls back to trafilatura or httpx on failure. The plugin is compatible with clients such as DSH, Claude Code, and Cursor, but users should pin the Playwright version and respect concurrency limits.