apifyRun and manage Apify Actors via REST API to scrape websites, crawl pages, extract data, and retrieve results from Apify datasets and key-value stores.
Install via ClawdBot CLI:
clawdbot install bmestanov/apifyRun any of the 17,000+ Actors on Apify Store and retrieve structured results via the REST API.
Full OpenAPI spec: openapi.json
All requests need the APIFY_TOKEN env var. Use it as a Bearer token:
-H "Authorization: Bearer $APIFY_TOKEN"Base URL: https://api.apify.com
Search the Apify Store by keyword:
curl -s "https://api.apify.com/v2/store?search=web+scraper&limit=5" \
-H "Authorization: Bearer $APIFY_TOKEN" | jq '.data.items[] | {name: (.username + "/" + .name), title, description}'Actors are identified by username~name (tilde) in API paths, e.g. apify~web-scraper.
Before running an Actor, fetch its default build to get the README (usage docs) and input schema (expected JSON fields):
curl -s "https://api.apify.com/v2/acts/apify~web-scraper/builds/default" \
-H "Authorization: Bearer $APIFY_TOKEN" | jq '.data | {readme, inputSchema}'inputSchema is a JSON-stringified object — parse it to see required/optional fields, types, defaults, and descriptions. Use this to construct valid input for the run.
You can also get the Actor's per-build OpenAPI spec (no auth required):
curl -s "https://api.apify.com/v2/acts/apify~web-scraper/builds/default/openapi.json"Start the Actor and get the run object back immediately:
curl -s -X POST "https://api.apify.com/v2/acts/apify~web-scraper/runs" \
-H "Authorization: Bearer $APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"startUrls":[{"url":"https://example.com"}],"maxPagesPerCrawl":10}'Response includes data.id (run ID), data.defaultDatasetId, data.status.
Optional query params: ?timeout=300&memory=4096&maxItems=100&waitForFinish=60
waitForFinish (0-60): seconds the API waits before returning. Useful to avoid polling for short runs.curl -s "https://api.apify.com/v2/actor-runs/RUN_ID?waitForFinish=60" \
-H "Authorization: Bearer $APIFY_TOKEN" | jq '.data | {status, defaultDatasetId}'Terminal statuses: SUCCEEDED, FAILED, ABORTED, TIMED-OUT.
Dataset items (most common — structured scraped data):
curl -s "https://api.apify.com/v2/datasets/DATASET_ID/items?clean=true&limit=100" \
-H "Authorization: Bearer $APIFY_TOKEN"Or directly from the run (shortcut — same parameters):
curl -s "https://api.apify.com/v2/actor-runs/RUN_ID/dataset/items?clean=true&limit=100" \
-H "Authorization: Bearer $APIFY_TOKEN"Params: format (json|csv|jsonl|xml|xlsx|rss), fields, omit, limit, offset, clean, desc.
Key-value store record (screenshots, HTML, OUTPUT):
curl -s "https://api.apify.com/v2/key-value-stores/STORE_ID/records/OUTPUT" \
-H "Authorization: Bearer $APIFY_TOKEN"Run log:
curl -s "https://api.apify.com/v2/logs/RUN_ID" \
-H "Authorization: Bearer $APIFY_TOKEN"For Actors that finish within 300 seconds, get dataset items in one call:
curl -s -X POST "https://api.apify.com/v2/acts/apify~web-scraper/run-sync-get-dataset-items?timeout=120" \
-H "Authorization: Bearer $APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"startUrls":[{"url":"https://example.com"}],"maxPagesPerCrawl":5}'Returns the dataset items array directly (not wrapped in data). Returns 408 if the run exceeds 300s.
Alternative: /run-sync returns the KVS OUTPUT record instead of dataset items.
curl -s -X POST "https://api.apify.com/v2/acts/apify~web-scraper/run-sync-get-dataset-items?timeout=120" \
-H "Authorization: Bearer $APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"startUrls":[{"url":"https://example.com"}],"maxPagesPerCrawl":20}'curl -s -X POST "https://api.apify.com/v2/acts/apify~google-search-scraper/run-sync-get-dataset-items?timeout=120" \
-H "Authorization: Bearer $APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"queries":"site:example.com openai","maxPagesPerQuery":1}'# 1. Start
RUN=$(curl -s -X POST "https://api.apify.com/v2/acts/apify~web-scraper/runs?waitForFinish=60" \
-H "Authorization: Bearer $APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"startUrls":[{"url":"https://example.com"}],"maxPagesPerCrawl":500}')
RUN_ID=$(echo "$RUN" | jq -r '.data.id')
# 2. Poll until done
while true; do
STATUS=$(curl -s "https://api.apify.com/v2/actor-runs/$RUN_ID?waitForFinish=60" \
-H "Authorization: Bearer $APIFY_TOKEN" | jq -r '.data.status')
echo "Status: $STATUS"
case "$STATUS" in SUCCEEDED|FAILED|ABORTED|TIMED-OUT) break;; esac
done
# 3. Fetch results
curl -s "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?clean=true" \
-H "Authorization: Bearer $APIFY_TOKEN"curl -s -X POST "https://api.apify.com/v2/actor-runs/RUN_ID/abort" \
-H "Authorization: Bearer $APIFY_TOKEN"Some Actors require a monthly subscription before they can be run. If the API returns a permissions or payment error for an Actor, ask the user to manually subscribe via the Apify Console:
https://console.apify.com/actors/ACTOR_IDReplace ACTOR_ID with the Actor's ID (e.g. AhEsMsQyLfHyMLaxz). The user needs to click Start on that page to activate the subscription. Most rental Actors offer a free trial period set by the developer.
You can get the Actor ID from the store search response (data.items[].id) or from GET /v2/acts/username~name (data.id).
APIFY_TOKEN missing or invalid.username~name format (tilde, not slash). Browse https://apify.com/store.GET /v2/logs/RUN_ID for details.Generated Mar 1, 2026
E-commerce retailers can use Apify Actors to scrape competitor websites daily, tracking price changes, stock availability, and promotional offers. This enables dynamic pricing strategies and inventory management decisions based on real-time market data.
Property agencies and investors can crawl multiple listing sites to collect property details, prices, locations, and amenities. This data helps identify market trends, valuation patterns, and investment opportunities across different neighborhoods.
Recruitment firms and HR departments can scrape job boards to analyze salary ranges, required skills, and hiring trends across industries. This supports talent acquisition strategies, compensation benchmarking, and workforce planning initiatives.
Marketing agencies can use specialized Actors to collect public social media posts about brands, products, or campaigns. This provides sentiment analysis data to measure campaign effectiveness and identify emerging customer concerns.
Researchers and academic institutions can automate the collection of scientific publications, news articles, or government reports for systematic reviews and meta-analyses. This saves hundreds of manual hours in data gathering for literature reviews.
Companies can build automated data pipelines using Apify Actors to collect, clean, and deliver structured web data to clients on a subscription basis. This creates recurring revenue streams by providing clients with valuable datasets they need for business intelligence.
Agencies can offer bespoke scraping services by configuring and running specific Actors for clients' unique data needs. This includes setting up scheduled runs, data transformation, and integration with clients' existing systems through API delivery.
Entrepreneurs can create specialized platforms that aggregate data from multiple sources using various Actors, then provide analytics dashboards and insights to specific industries. This adds value through data visualization, trend analysis, and predictive modeling.
💬 Integration Tip
Always test Actors with small datasets first to validate output format, and implement proper error handling for rate limits and website structure changes that might break scrapers.
Automates browser interactions for web testing, form filling, screenshots, and data extraction. Use when the user needs to navigate websites, interact with w...
Playwright-based web scraping OpenClaw Skill with anti-bot protection. Successfully tested on complex sites like Discuss.com.hk.
Browser automation and web scraping with Playwright. Forms, screenshots, data extraction. Works standalone or via MCP. Testing included.
Performs deep scraping of complex sites like YouTube using containerized Crawlee, extracting validated, ad-free transcripts and content as JSON output.
Automate web tasks like form filling, data scraping, testing, monitoring, and scheduled jobs with multi-browser support and retry mechanisms.
Web scraping and content comprehension agent — multi-strategy extraction with cascade fallback, news detection, boilerplate removal, structured metadata, and...