haibo-jina-readerExtract clean, readable markdown content from any URL using Jina Reader API. Use when you need to fetch and parse web pages without dealing with HTML, JavaScript rendering, or paywalls. Ideal for research, article summarization, content analysis, and working with search results from tavily-search, web_search, or searxng skills.
Install via ClawdBot CLI:
clawdbot install smile-xuc/haibo-jina-readerJina Reader provides clean markdown extraction from any URL, bypassing HTML complexity, JavaScript rendering, and many paywalls. It returns structured text content with metadata (title, URL, published time) that's perfect for AI analysis.
scripts/jina-reader.py <url>scripts/jina-reader.py <url> --format jsonscripts/jina-reader.py <url> -o output.mdExtract clean markdown from any URL:
scripts/jina-reader.py https://example.com/articleReturns: Full markdown content including title, metadata headers, and structured text.
Use when: You need readable text from a webpage for summarization, analysis, or content processing.
Get structured data with metadata:
scripts/jina-reader.py https://example.com/article --format jsonReturns:
{
"status": "success",
"metadata": {
"title": "Article Title",
"url": "https://example.com/article",
"published": "Mon, 10 Feb 2026 12:00:00 GMT"
},
"content": "Markdown content..."
}Use when: You need programmatic access to metadata or want to integrate with other tools.
For simple one-liners:
scripts/jina-reader.sh https://example.com/articleReturns: Raw markdown content directly to stdout.
Use when: Quick extraction without arguments or when piping to other commands.
When using tavily-search, web_search, or searxng skills:
# Example workflow
URL="https://example.com/article"
scripts/jina-reader.py "$URL" --format json | jq -r '.content'Extract from multiple URLs:
for url in $(cat urls.txt); do
scripts/jina-reader.py "$url" -o "output/$(basename $url).md"
donePipe extracted content to analysis tools:
scripts/jina-reader.py https://example.com/article | wc -w
scripts/jina-reader.py https://example.com/article | grep -i "keyword"jina-reader.py)url (required): The URL to extract content from-f, --format: Output format - markdown or json (default: markdown)-t, --timeout: Request timeout in seconds (default: 30)-o, --output: Save output to file instead of stdoutjina-reader.sh)url (required): The URL to extract content from-t for slow-loading pagesscripts/jina-reader.py <url> -t 60 # Increase timeoutThe tool auto-prepends https:// if missing. Use fully qualified URLs for reliability.
Some pages may block scraping. Try the shell script as fallback, or verify the URL is accessible.
Full-featured Python tool with JSON output, metadata extraction, and file saving.
Lightweight shell script for quick markdown extraction.
Generated Mar 1, 2026
Extract content from competitor websites, industry reports, and news articles to analyze trends, product features, and market positioning. This enables businesses to gather structured data for benchmarking and strategic planning without manual web scraping.
Fetch and parse scholarly articles, research papers, and online publications into clean markdown for literature reviews, citation analysis, and data extraction. It simplifies content aggregation from diverse sources, aiding in hypothesis testing and knowledge synthesis.
Automatically extract articles from news websites, blogs, and RSS feeds to create curated newsletters, summaries, or content databases. This supports media companies and digital publishers in streamlining content workflows and enhancing reader engagement.
Retrieve and structure content from regulatory websites, legal documents, and public records for compliance checks, due diligence, and policy analysis. It helps legal teams efficiently monitor updates and extract relevant information without dealing with complex HTML.
Analyze web content from top-ranking pages to extract keywords, meta descriptions, and structural elements for SEO strategy. Marketers and SEO specialists can use this to benchmark performance and optimize their own content for search engines.
Offer a cloud-based API service with tiered pricing based on usage volume (e.g., requests per month). Target businesses needing scalable content extraction for research, analytics, or automation, with premium features like higher rate limits and priority support.
Provide on-premise or custom deployments for large organizations with strict data privacy or integration needs. Include dedicated support, custom features, and bulk processing capabilities, catering to industries like finance, healthcare, or government.
Offer a free tier with limited daily requests to attract individual developers and small teams, then monetize through paid API credits for higher usage. This model encourages adoption and upsells to power users and growing businesses.
๐ฌ Integration Tip
Combine jina-reader with search skills like tavily-search to first find relevant URLs, then extract and process content in batch workflows for efficient data pipelines.
Summarize URLs or files with the summarize CLI (web, PDFs, images, audio, YouTube).
AI-optimized web search via Tavily API. Returns concise, relevant results for AI agents.
This skill should be used when users need to search the web for information, find current content, look up news articles, search for images, or find videos. It uses DuckDuckGo's search API to return results in clean, formatted output (text, markdown, or JSON). Use for research, fact-checking, finding recent information, or gathering web resources.
Web search and content extraction via Brave Search API. Use for searching documentation, facts, or any web content. Lightweight, no browser required.
Search indexed Discord community discussions via Answer Overflow. Find solutions to coding problems, library issues, and community Q&A that only exist in Discord conversations.
Multi search engine integration with 17 engines (8 CN + 9 Global). Supports advanced search operators, time filters, site search, privacy engines, and WolframAlpha knowledge queries. No API keys required.