iran-chem-databasev2.9.0

Iran Chemical Database — an HTTrack-powered live crawling system that builds a dated, auditable, BEST-EFFORT index of confirmed and unresolved chemical offerings discovered in configured public Iranian supplier catalogues. Discovers and mirrors supplier websites with HTTrack (polite crawling, robots-aware), extracts catalogue entries from local mirrors (HTML/JSON-LD/PDF/Excel/DOCX/JSON-API), classifies with a configurable inclusion policy (research_only | lab_or_research | all_identifiable_catalogue) plus explicit structure-first organic classification, validates with RDKit/PubChem, and maintains a live PostgreSQL database with FastAPI + Streamlit interfaces. Coverage, crawl states and rejection reasons are measured and published — never claimed complete. Installation provides software, not a populated dataset. v2.6 adds a WooCommerce/WordPress REST + sitemap engine (public /wp-json/wc/store/v1/products), a fingerprint-annotated seed list (dead/parked/geo-blocked/WooCommerce per-URL), geo-block detection, and a fixed Celery dispatch path. Persian NLP support, Playwright + network-recording fallback for JS/API catalogues, Celery scheduling, Docker deployment. For academic procurement research.