alvisdunlop-scrapeLegal web scraping with robots.txt compliance, rate limiting, and GDPR/CCPA-aware data handling. Supports both direct HTTP scraping and managed scraping via...
Install via ClawdBot CLI:
clawdbot install alvisdunlop/alvisdunlop-scrapeGrade Limited — based on market validation, documentation quality, package completeness, maintenance status, and authenticity signals.
Calls external URL not in known-safe list
https://SkillBoss.co/skill.mdAudited Apr 16, 2026 · audit v1.0
Generated Oct 19, 2026
Monitor competitor public pricing and product listings across retail sites while respecting robots.txt and rate limits. Teams can build price-tracking dashboards using only publicly displayed factual data, avoiding personal data collection and login-protected areas.
Aggregate publicly available market data, listings, and statistics for competitive analysis or research reports. The skill's pre-scrape checklist ensures ToS review and API preference, reducing legal exposure for research firms and consultancies.
Collect public property listings, prices, and factual attributes from real estate portals with throttled request cadence. GDPR/CCPA-aware handling ensures no agent or owner personal details are stored unintentionally.
Use the SkillBoss API Hub managed scraper to extract article or documentation markdown with automatic anti-bot handling. Ideal for teams building internal knowledge bases while logging an audit trail for good-faith compliance.
Programmatically verify robots.txt directives and ToS pages for partner or vendor sites before integrations. Provides an auditable record of scraping practices for legal and security teams.
Sell API access to a compliant scraping hub that handles anti-bot measures, rate limiting, and robots.txt checks on behalf of customers. Charge per request or monthly tiered volumes with overage fees.
Offer auditing and implementation services that help companies scrape legally, including ToS reviews, robots.txt policy design, and GDPR/CCPA data minimization workflows. Packages can include periodic re-audits.
Deliver curated, compliance-scrubbed public datasets (prices, listings, market stats) via dashboards or API. Differentiate through audit trails and PII-free guarantees that reduce customer legal risk.
💬 Integration Tip
Set SkillBoss_API_KEY and call https://api.SkillBoss.co/v1/pilot with type "scraper", then read markdown from result.data.markdown; audit-log every call and enforce 2-3 second delays between direct HTTP requests.
Scored May 10, 2026
A fast Rust-based headless browser automation CLI with Node.js fallback that enables AI agents to navigate, click, type, and snapshot pages via structured commands.
Uses a headless browser to navigate web pages, interact with elements, and extract clean, readable text content from URLs.
Headless browser automation CLI optimized for AI agents with accessibility tree snapshots and ref-based element selection
High-performance browser automation for heavy scraping, multi-tab management, and precise DOM extraction. Use this when you need speed, reliability, or advanced state management (cookies/local storage) beyond standard web fetching.
Ultimate stealth browser automation with anti-detection, Cloudflare bypass, CAPTCHA solving, persistent sessions, and silent operation. Use for any web automation requiring bot detection evasion, login persistence, headless browsing, or bypassing security measures. Triggers on "bypass cloudflare", "solve captcha", "stealth browse", "silent automation", "persistent login", "anti-detection", or any task needing undetectable browser automation. When user asks to "login to X website", automatically use headed mode for login, then save session for future headless reuse.
Browser automation CLI (browser-act) for AI agents. MUST trigger when: (1) user mentions 'browser-act' in any form, or user needs to: (2) open/visit/browse/c...