scrapeLegal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling.
Install via ClawdBot CLI:
clawdbot install ivangdavila/scrapeGrade Fair — based on market validation, documentation quality, package completeness, maintenance status, and authenticity signals.
Calls external URL not in known-safe list
https://example.com/productsAudited Apr 16, 2026 · audit v1.0
Generated Mar 1, 2026
Scrape public pricing data from e-commerce sites like Amazon or Walmart to track competitor prices. Ensure compliance by checking robots.txt, using rate limiting, and avoiding personal data to support dynamic pricing strategies.
Collect property details such as prices, locations, and features from public real estate websites like Zillow. Follow legal guidelines by respecting robots.txt, minimizing data storage, and stripping any PII to create aggregated market reports.
Scrape job postings from career sites like LinkedIn or Indeed to analyze hiring trends and skill demands. Comply by using a proper User-Agent, avoiding login-protected data, and ensuring no personal information is collected for industry insights.
Gather public factual data from educational or government websites for research purposes, such as climate statistics or economic indicators. Adhere to legal boundaries by checking Terms of Service, implementing rate limits, and maintaining an audit trail for transparency.
Extract headlines and article summaries from news websites to track media trends or sentiment analysis. Ensure compliance by verifying robots.txt, using session reuse to reduce server load, and avoiding republishing copyrighted content.
Provide cleaned and structured scraped data to clients via subscription or one-time sales. Focus on legal compliance by using APIs when available, stripping PII, and maintaining audit trails to build trust and avoid violations.
Offer analytics dashboards based on scraped data, such as competitor insights or trend reports. Generate revenue through SaaS subscriptions by ensuring data is collected ethically with rate limiting and robots.txt compliance.
Develop and sell tailored scraping scripts or services for specific client needs, like real estate or retail monitoring. Monetize through project-based fees by emphasizing legal adherence, such as GDPR-aware data handling and ToS checks.
💬 Integration Tip
Integrate this skill by first checking robots.txt and Terms of Service programmatically, then using rate-limited requests with proper User-Agents to avoid legal issues and server strain.
Scored Apr 19, 2026
A fast Rust-based headless browser automation CLI with Node.js fallback that enables AI agents to navigate, click, type, and snapshot pages via structured commands.
Uses a headless browser to navigate web pages, interact with elements, and extract clean, readable text content from URLs.
Headless browser automation CLI optimized for AI agents with accessibility tree snapshots and ref-based element selection
通过已登录的Edge或Chrome浏览器,利用Chrome DevTools Protocol执行JS渲染页面的导航、点击、截图及数据提取等自动化操作。
Send and receive SMS/RCS via Google Messages web interface (messages.google.com). Use when asked to "send a text", "check texts", "SMS", "text message", "Google Messages", or forward incoming texts to other channels.
High-performance browser automation for heavy scraping, multi-tab management, and precise DOM extraction. Use this when you need speed, reliability, or advanced state management (cookies/local storage) beyond standard web fetching.