xml-sitemap-generatorGenerate XML sitemaps by crawling a website or scanning local files. Auto-discovers pages via link extraction. Supports local HTML/MD file scanning with last...
Install via ClawdBot CLI:
clawdbot install charlie-morrison/xml-sitemap-generatorGrade Fair — based on market validation, documentation quality, package completeness, maintenance status, and authenticity signals.
Calls external URL not in known-safe list
https://example.comAudited Apr 16, 2026 · audit v1.0
Generated Oct 24, 2026
A marketing team finishes building a new multi-page website and needs a sitemap.xml and robots.txt for search engine submission. They run the crawler against staging and production URLs to discover all pages via BFS link discovery, then save both files to the web root before the official launch.
A developer is migrating hundreds of hand-coded HTML and Markdown files to a new hosting platform. Using local mode with --base-url, they scan the ./public directory to enumerate every page and auto-derive lastmod dates from file mtimes, producing an accurate sitemap without touching the live server.
An SEO consultant crawls a client's online store to generate an up-to-date sitemap and verify crawlable pages. With --max-pages raised above the default and --verbose enabled, they monitor crawl progress and identify any link gaps or redirects that need fixing before submitting the sitemap to Google Search Console.
A technical writing team maintains a docs portal with HTML and Markdown sources. They run the tool in local mode during CI to regenerate sitemap.xml automatically, ensuring every new doc page is discoverable by search engines while the smart filtering skips embedded images and PDFs.
A web agency automates sitemap and robots.txt generation as a final step in every client deployment. The crawler runs against the live domain to confirm all expected pages resolve, and --robots writes a robots.txt with the sitemap reference so each launch follows the same SEO-ready process.
Website builders and CMS platforms integrate the sitemap generator so customers automatically get an up-to-date sitemap.xml on every publish. It removes a common SEO headache for non-technical users and creates a differentiator against competitors.
Agencies use the tool as a rapid, dependency-free diagnostics step in paid SEO audits, producing sitemaps and robots.txt for clients during engagement. The tool shortens audit turnaround and lets consultants focus on recommendation work.
A hosting or CI/CD provider bakes sitemap generation into its build pipeline, crawling or scanning the generated site and publishing sitemap.xml and robots.txt with every deploy. This value-added automation increases retention and upsell opportunities.
💬 Integration Tip
The script is stdlib-only Python 3.6+, so it can be dropped into any CI pipeline or build step without installing dependencies; use --local mode with --base-url for pre-deployment runs and crawl the live URL once to validate that every expected page is discoverable.
Scored Jun 21, 2026
Fetch and read transcripts from YouTube videos. Use when you need to summarize a video, answer questions about its content, or extract information from it.
Monitor RSS and Atom feeds for content research. Track blogs, news sites, newsletters, and any feed source. Use when monitoring competitors, tracking industr...
用 MinerU API 解析 PDF/Word/PPT/图片为 Markdown，支持公式、表格、OCR。适用于论文解析、文档提取。
Provides a personalized morning report with today's reminders, undone Notion tasks, and vault storage summary for daily planning.
Extract text from PDFs with OCR support. Perfect for digitizing documents, processing invoices, or analyzing content. Zero dependencies required.
Fetch scheduled economic events and data releases from the FMP API for specified dates, filtering by impact, country, and type, and output a chronological ma...