scrapeLegal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling.
Install via ClawdBot CLI:
clawdbot install ivangdavila/scrapeGrade Fair — based on market validation, documentation quality, package completeness, maintenance status, and authenticity signals.
Calls external URL not in known-safe list
https://example.com/productsAudited Apr 16, 2026 · audit v1.0
Generated Mar 1, 2026
Scrape public pricing data from e-commerce sites like Amazon or Walmart to track competitor prices. Ensure compliance by checking robots.txt, using rate limiting, and avoiding personal data to support dynamic pricing strategies.
Collect property details such as prices, locations, and features from public real estate websites like Zillow. Follow legal guidelines by respecting robots.txt, minimizing data storage, and stripping any PII to create aggregated market reports.
Scrape job postings from career sites like LinkedIn or Indeed to analyze hiring trends and skill demands. Comply by using a proper User-Agent, avoiding login-protected data, and ensuring no personal information is collected for industry insights.
Gather public factual data from educational or government websites for research purposes, such as climate statistics or economic indicators. Adhere to legal boundaries by checking Terms of Service, implementing rate limits, and maintaining an audit trail for transparency.
Extract headlines and article summaries from news websites to track media trends or sentiment analysis. Ensure compliance by verifying robots.txt, using session reuse to reduce server load, and avoiding republishing copyrighted content.
Provide cleaned and structured scraped data to clients via subscription or one-time sales. Focus on legal compliance by using APIs when available, stripping PII, and maintaining audit trails to build trust and avoid violations.
Offer analytics dashboards based on scraped data, such as competitor insights or trend reports. Generate revenue through SaaS subscriptions by ensuring data is collected ethically with rate limiting and robots.txt compliance.
Develop and sell tailored scraping scripts or services for specific client needs, like real estate or retail monitoring. Monetize through project-based fees by emphasizing legal adherence, such as GDPR-aware data handling and ToS checks.
💬 Integration Tip
Integrate this skill by first checking robots.txt and Terms of Service programmatically, then using rate-limited requests with proper User-Agents to avoid legal issues and server strain.
Scored Apr 18, 2026
A fast Rust-based headless browser automation CLI with Node.js fallback that enables AI agents to navigate, click, type, and snapshot pages via structured commands.
Automate web browser interactions using natural language via CLI commands. Use when the user asks to browse websites, navigate web pages, extract data from websites, take screenshots, fill forms, click buttons, or interact with web applications.
Automates browser interactions for web testing, form filling, screenshots, and data extraction. Use when the user needs to navigate websites, interact with w...
Headless browser automation CLI optimized for AI agents with accessibility tree snapshots and ref-based element selection
Playwright-based web scraping OpenClaw Skill with anti-bot protection. Successfully tested on complex sites like Discuss.com.hk.
Uses a headless browser to navigate web pages, interact with elements, and extract clean, readable text content from URLs.