memory-pipelineComplete agent memory + performance system. Extracts structured facts, builds knowledge graphs, generates briefings, and enforces execution discipline via pre-game routines, tool policies, result compression, and after-action reviews. Includes external knowledge ingestion (ChatGPT exports, etc.) into searchable memory. Use when working on memory management, briefing generation, knowledge consolidation, external data ingestion, agent consistency, or improving execution quality across sessions.
Install via ClawdBot CLI:
clawdbot install joe-rlo/memory-pipelineGive your AI agent a memory that actually works.
AI agents wake up blank every session. Memory Pipeline fixes that ā it extracts what matters from past conversations, connects the dots, and generates a daily briefing so your agent starts each session primed instead of clueless.
| Component | When it runs | What it does |
|-----------|-------------|--------------|
| Extract | Between sessions | Pulls structured facts (decisions, preferences, learnings) from daily notes and transcripts |
| Link | Between sessions | Builds a knowledge graph ā connects related facts, flags contradictions |
| Brief | Between sessions | Generates a compact BRIEFING.md loaded at session start |
| Ingest | On demand | Imports external knowledge (ChatGPT exports, etc.) into searchable memory |
| Performance Hooks | During sessions | Pre-game briefing injection, tool discipline, output compression, after-action review |
Most "memory" solutions are just vector search over chat logs. This is a cognitive architecture ā inspired by how human memory actually works:
clawdhub install memory-pipelinebash skills/memory-pipeline/scripts/setup.shThe setup script will detect your workspace, check dependencies (Python 3 + any LLM API key), create the memory/ directory, and run the full pipeline.
OPENAI_API_KEY or ~/.config/openai/api_key)ANTHROPIC_API_KEY or ~/.config/anthropic/api_key)GEMINI_API_KEY or ~/.config/gemini/api_key)# Full pipeline
python3 skills/memory-pipeline/scripts/memory-extract.py
python3 skills/memory-pipeline/scripts/memory-link.py
python3 skills/memory-pipeline/scripts/memory-briefing.pyAdd to your HEARTBEAT.md for daily automatic runs:
### Daily Memory Pipeline
- **Frequency:** Once per day (morning)
- **Action:** Run the memory pipeline:
1. `python3 skills/memory-pipeline/scripts/memory-extract.py`
2. `python3 skills/memory-pipeline/scripts/memory-link.py`
3. `python3 skills/memory-pipeline/scripts/memory-briefing.py`Already have years of conversations in ChatGPT? Import them so your agent knows what you know.
# 1. Export from ChatGPT: Settings ā Data Controls ā Export Data
# 2. Drop the zip in your workspace
# 3. Run:
python3 skills/memory-pipeline/scripts/ingest-chatgpt.py ~/imports/chatgpt-export.zip
# Preview first (recommended):
python3 skills/memory-pipeline/scripts/ingest-chatgpt.py ~/imports/chatgpt-export.zip --dry-runWhat it does:
--min-turns, --min-length)EXCLUDE_PATTERNS to skip unwanted topics)memory/knowledge/chatgpt/Options:
--dry-run ā Preview without writing files--keep-all ā Skip all filtering--min-turns N ā Minimum user messages to keep (default: 2)--min-length N ā Minimum total characters (default: 200)The pattern is extensible. Create ingest-, parse the format, write markdown to memory/knowledge/. The indexer handles the rest.
Script: memory-extract.py
Reads daily notes (memory/YYYY-MM-DD.md) and session transcripts, then uses an LLM to extract structured facts:
{"type": "decision", "content": "Use Rust for the backend", "subject": "Project Architecture", "confidence": 0.9}
{"type": "preference", "content": "Prefers Google Drive over Notion", "subject": "Tools", "confidence": 0.95}Output: memory/extracted.jsonl
Script: memory-link.py
Takes extracted facts and builds a knowledge graph:
Output: memory/knowledge-graph.json + memory/knowledge-summary.md
Script: memory-briefing.py
Generates a compact daily briefing (< 2000 chars) combining:
SOUL.md)USER.md)Output: BRIEFING.md (workspace root)
Four lifecycle hooks that enforce execution discipline during sessions. Based on a principle from performance psychology: separate preparation from execution.
User Message ā Agent Loop
āāā before_agent_start ā Briefing packet (memory + checklist)
āāā before_tool_call ā Policy enforcement (deny list)
āāā tool_result_persist ā Output compression (prevent context bloat)
āāā agent_end ā After-action review (durable notes){
"enabled": true,
"briefing": {
"maxChars": 6000,
"checklist": [
"Restate the task in one sentence.",
"List constraints and success criteria.",
"Retrieve only the minimum relevant memory.",
"Prefer tools over guessing when facts matter."
],
"memoryFiles": ["memory/IDENTITY.md", "memory/PROJECTS.md"]
},
"tools": {
"deny": ["dangerous_tool"],
"maxToolResultChars": 12000
},
"afterAction": {
"writeMemoryFile": "memory/AFTER_ACTION.md",
"maxBullets": 8
}
}| Hook | What it does |
|------|-------------|
| before_agent_start | Loads memory files, builds bounded briefing packet, injects into system prompt |
| before_tool_call | Checks tool against deny list, prevents unsafe calls |
| tool_result_persist | Head (60%) + tail (30%) compression of large results |
| agent_end | Appends session summary to memory file with tools used and outcomes |
| File | Location | Purpose |
|------|----------|---------|
| BRIEFING.md | Workspace root | Daily context cheat sheet |
| extracted.jsonl | memory/ | All extracted facts (append-only) |
| knowledge-graph.json | memory/ | Full graph with embeddings and links |
| knowledge-summary.md | memory/ | Human-readable graph summary |
| knowledge/chatgpt/*.md | memory/ | Ingested ChatGPT conversations |
memory-extract.py to focus on different fact typesmemory-link.py (default: 0.3)EXCLUDE_PATTERNS in ingest-chatgpt.py for topic exclusion| Problem | Fix |
|---------|-----|
| No facts extracted | Check that daily notes or transcripts exist; verify API key |
| Low-quality links | Add OpenAI key for embedding-based similarity; adjust threshold |
| Briefing too long | Reduce facts in template or let LLM generation handle it (auto-constrained to 2000 chars) |
Generated Mar 1, 2026
An executive uses the Memory Pipeline to maintain continuity across daily briefings, ensuring their AI assistant remembers key decisions, preferences, and project updates from previous interactions. This improves productivity by eliminating repetitive context-setting and enabling proactive support based on historical data.
A customer support team integrates the Memory Pipeline to track recurring issues, user preferences, and resolution histories across sessions. This allows agents to provide personalized, efficient support without manually reviewing past tickets, enhancing customer satisfaction and reducing response times.
A researcher employs the Memory Pipeline to consolidate findings, hypotheses, and external knowledge sources (like ChatGPT exports) into a searchable knowledge graph. This aids in connecting disparate data points, generating insights, and maintaining project continuity over long-term studies.
A development team uses the Memory Pipeline to document technical decisions, coding preferences, and project milestones extracted from daily notes and meetings. The generated briefings help onboard new members and ensure consistency in tool usage and architectural choices across sessions.
A content creator leverages the Memory Pipeline to store style guidelines, audience preferences, and past campaign results. This enables AI agents to produce content that aligns with brand voice and historical performance, improving coherence and engagement across marketing efforts.
Offer the Memory Pipeline as a cloud-based service with tiered pricing based on usage volume and features like advanced analytics or custom integrations. Revenue is generated through monthly or annual subscriptions, targeting businesses seeking to enhance AI agent performance and team collaboration.
Provide professional services to help organizations implement and customize the Memory Pipeline for specific workflows, such as integrating with existing CRM or project management tools. Revenue comes from one-time setup fees and ongoing support contracts.
Distribute a basic version of the Memory Pipeline for free to individual users, with limitations on data volume or advanced features. Monetize through paid upgrades for enterprises, including priority support, enhanced security, and bulk external knowledge ingestion capabilities.
š¬ Integration Tip
Start by automating the pipeline via HEARTBEAT.md for daily runs to ensure consistent memory updates, and gradually incorporate external knowledge ingestion to expand the agent's contextual awareness.
Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Clau...
Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.
Search and analyze your own session logs (older/parent conversations) using jq.
Typed knowledge graph for structured agent memory and composable skills. Use when creating/querying entities (Person, Project, Task, Event, Document), linking related objects, enforcing constraints, planning multi-step actions as graph transformations, or when skills need to share state. Trigger on "remember", "what do I know about", "link X to Y", "show dependencies", entity CRUD, or cross-skill data access.
Ultimate AI agent memory system for Cursor, Claude, ChatGPT & Copilot. WAL protocol + vector search + git-notes + cloud backup. Never lose context again. Vibe-coding ready.
Headless browser automation CLI optimized for AI agents with accessibility tree snapshots and ref-based element selection