python-web-scraperPython web scraping toolkit for data extraction, pagination handling, anti-blocking techniques, Selenium for JavaScript-heavy sites, and structured output (J...
Install via ClawdBot CLI:
clawdbot install ericlooi504/python-web-scraperGrade Fair — based on market validation, documentation quality, package completeness, maintenance status, and authenticity signals.
Sends data to undocumented external endpoint (potential exfiltration)
post → https://example.com/loginCalls external URL not in known-safe list
https://example.comAI Analysis
The skill definition includes example code that posts credentials to 'https://example.com/login', which is a placeholder domain and not a real endpoint, so no actual exfiltration occurs. However, the skill's purpose is legitimate web scraping, and the example is clearly instructional, not malicious. The low risk is due to the potential for misuse if users replace placeholders with real credentials, but the skill itself does not contain hidden instructions or obfuscation.
Audited May 10, 2026 · audit v1.0
Generated Nov 18, 2026
E-commerce businesses need to track competitor product prices and availability across multiple retail sites. Using pagination handling and CSS selectors, the scraper can collect pricing data from category pages and product listings, then output structured JSON/CSV for analysis. Anti-blocking techniques like rotating user-agents and delays help avoid IP bans during frequent checks.
Analysts and consultants gather large volumes of public data from news sites, blogs, and directories for market trend analysis. The scraper extracts article titles, publication dates, and content snippets from paginated news feeds or search results. Output is formatted into CSV for easy import into spreadsheets or BI tools.
Real estate agencies and property portals scrape listings from multiple websites to build a comprehensive database of available properties. The toolkit handles dynamic content on JavaScript-heavy sites using Selenium, and manages pagination to capture all listings. Extracted data includes price, location, and property details, exported as JSON for further processing.
Recruiters and HR teams monitor job boards for new postings relevant to their industry. The scraper extracts job titles, companies, and descriptions from paginated job listing pages, and can handle JavaScript-rendered content on modern job boards. Data is output to CSV for tracking and analysis of hiring trends.
Researchers collect data from public websites such as government statistics pages, academic repositories, or social media for analysis. The scraper handles pagination and JavaScript-rendered content to extract structured data. Ethical scraping practices are emphasized, including checking robots.txt and adding delays to avoid overloading servers.
A business that uses web scraping to collect and sell structured data feeds or datasets to clients. They scrape e-commerce prices, real estate listings, or job postings and provide cleaned data via API or subscription. Revenue comes from recurring subscription fees based on data volume or frequency.
A SaaS platform that aggregates and analyzes scraped data to provide competitive intelligence, pricing insights, or trend reports. The scraping toolkit powers the data collection backend, while the platform offers dashboards and alerts. Revenue is generated through tiered SaaS subscriptions.
A consulting or freelance business that builds custom web scrapers for clients who need specific data extraction but lack technical skills. The agency uses the toolkit to quickly deliver projects, handling anti-blocking and pagination. Revenue is project-based or retainer-based for ongoing data collection.
💬 Integration Tip
Integrate the scraper with scheduling tools like cron or Airflow for recurring data collection, and use the JSON output to feed directly into databases or analytics platforms. Always implement rate limiting and respect robots.txt to avoid legal issues.
Scored Nov 18, 2026
A fast Rust-based headless browser automation CLI with Node.js fallback that enables AI agents to navigate, click, type, and snapshot pages via structured commands.
Uses a headless browser to navigate web pages, interact with elements, and extract clean, readable text content from URLs.
Headless browser automation CLI optimized for AI agents with accessibility tree snapshots and ref-based element selection
High-performance browser automation for heavy scraping, multi-tab management, and precise DOM extraction. Use this when you need speed, reliability, or advanced state management (cookies/local storage) beyond standard web fetching.
Ultimate stealth browser automation with anti-detection, Cloudflare bypass, CAPTCHA solving, persistent sessions, and silent operation. Use for any web automation requiring bot detection evasion, login persistence, headless browsing, or bypassing security measures. Triggers on "bypass cloudflare", "solve captcha", "stealth browse", "silent automation", "persistent login", "anti-detection", or any task needing undetectable browser automation. When user asks to "login to X website", automatically use headed mode for login, then save session for future headless reuse.
Browser automation CLI (browser-act) for AI agents. MUST trigger when: (1) user mentions 'browser-act' in any form, or user needs to: (2) open/visit/browse/c...