scrapeLegal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling.
Install via ClawdBot CLI:
clawdbot install ivangdavila/scrapeGrade Fair — based on market validation, documentation quality, package completeness, maintenance status, and authenticity signals.
Calls external URL not in known-safe list
https://example.com/productsAudited Apr 16, 2026 · audit v1.0
Generated Mar 1, 2026
Scrape public pricing data from e-commerce sites like Amazon or Walmart to track competitor prices. Ensure compliance by checking robots.txt, using rate limiting, and avoiding personal data to support dynamic pricing strategies.
Collect property details such as prices, locations, and features from public real estate websites like Zillow. Follow legal guidelines by respecting robots.txt, minimizing data storage, and stripping any PII to create aggregated market reports.
Scrape job postings from career sites like LinkedIn or Indeed to analyze hiring trends and skill demands. Comply by using a proper User-Agent, avoiding login-protected data, and ensuring no personal information is collected for industry insights.
Gather public factual data from educational or government websites for research purposes, such as climate statistics or economic indicators. Adhere to legal boundaries by checking Terms of Service, implementing rate limits, and maintaining an audit trail for transparency.
Extract headlines and article summaries from news websites to track media trends or sentiment analysis. Ensure compliance by verifying robots.txt, using session reuse to reduce server load, and avoiding republishing copyrighted content.
Provide cleaned and structured scraped data to clients via subscription or one-time sales. Focus on legal compliance by using APIs when available, stripping PII, and maintaining audit trails to build trust and avoid violations.
Offer analytics dashboards based on scraped data, such as competitor insights or trend reports. Generate revenue through SaaS subscriptions by ensuring data is collected ethically with rate limiting and robots.txt compliance.
Develop and sell tailored scraping scripts or services for specific client needs, like real estate or retail monitoring. Monetize through project-based fees by emphasizing legal adherence, such as GDPR-aware data handling and ToS checks.
💬 Integration Tip
Integrate this skill by first checking robots.txt and Terms of Service programmatically, then using rate-limited requests with proper User-Agents to avoid legal issues and server strain.
Scored Apr 19, 2026
A fast Rust-based headless browser automation CLI with Node.js fallback that enables AI agents to navigate, click, type, and snapshot pages via structured commands.
Browser automation via Playwright MCP server. Navigate websites, click elements, fill forms, extract data, take screenshots, and perform full browser automation workflows.
Performs deep scraping of complex sites like YouTube using containerized Crawlee, extracting validated, ad-free transcripts and content as JSON output.
Extract structured information from websites using web_fetch for simple pages and browser automation for dynamic sites, login-gated flows, pagination, infini...
OpenClaw skill for the agent-browser CLI (Rust-based with Node.js fallback) enabling AI-friendly web automation with snapshots, refs, and structured commands.
A modern text-based browser. Renders web pages in the terminal using headless Firefox.