Automated knowledge harvesting for AI engineers. Scrapes. Scores. Commits. Every 3 hours. Zero manual effort.
Overview · How It Works · Architecture · Quick Start · Vault Stats · Contributing
📌 Overview
Most AI knowledge bases go stale the moment you stop updating them. Vybe Intelligence Vault doesn't — it runs itself.
A GitHub Actions pipeline wakes up every 3 hours, discovers emerging AI/ML resources, evaluates them with an LLM scoring engine, and commits the ranked results back into the repo. No human in the loop. No manual curation.
The result: a self-reinforcing knowledge graph of 5,928 indexed resources spanning AI agents, RAG architectures, MCP servers, and modern web tooling — always current, always queryable by local agents via an HTTP gateway.
Built for: AI engineers who want a living knowledge base they can plug into agentic workflows, not a static awesome-list that someone forked two years ago.
⚙️ How It Works
Every 3 hours:
GitHub Actions Cron
│
▼
evaluate_repo.py ← discovers candidate resources from configured topics
│
▼
LLM Scoring Engine ← qwen2.5:14b (local) or cloud fallback
│ scores: quality, rag_relevance, tech_stack match
▼
vault-core/ ← ranked .md files committed back to repo
│
├─▶ rebuild-index.yml ← triggers on push
│ │
│ ▼
│ nomic-embed-text ← local Ollama embeddings
│ │
│ ▼
│ vault-index.json ← semantic node/edge graph (cosine sim > 0.75)
│ │
│ ▼
│ React 3D Map ← WebGL intelligence visualization (SWR polling)
│
└─▶ Orchestrator :3456 ← MCP gateway for agent context injection
Scoring Pipeline
Each resource is evaluated across 4 dimensions:
| Signal | Method |
|---|---|
| Content quality | LLM pass via qwen2.5:14b / cloud fallback |
| RAG relevance | Keyword + semantic scoring |
| Community velocity | Stars delta, fork rate |
| Tech stack match | Tag overlap with config.yaml topics |
Semantic Graph
Embeddings via nomic-embed-text (Ollama). Two nodes are linked if:
- Cosine similarity
> 0.75→similar_to - Shared tech stack →
depends_on - Same category + shared tags →
references
Edge weight = cosine_sim + (shared_tags × 0.08)
MCP Gateway
HTTP bridge on :3456. Send a vault file path → receive a clean, LLM-formatted context block. Agents can pull any resource into their context window without reading the filesystem directly.
# Example agent request
curl -X POST http://localhost:3456/inject \
-d '{"path": "ai/agents/tool-use-patterns.md"}'
🏗 Architecture
graph TD
A[⏰ Cron / Dispatch Trigger] -->|every 3h| B(evaluate_repo.py)
B -->|LLM score| C{Decision Engine}
C -->|pass threshold| D[vault-core/]
C -->|reject| X[❌ Discarded]
D -->|git push| E(rebuild-index.yml)
E -->|nomic-embed-text| F[vault-index.json]
F -->|SWR poll| G[���� React 3D Map]
H[🤖 AI Agent] -->|MCP request| I[Orchestrator :3456]
I -->|read + format| D
style A fill:#1f2937,color:#e5e7eb
style D fill:#111827,color:#e5e7eb
style G fill:#1f2937,color:#e5e7eb
style H fill:#374151,color:#e5e7eb
Key Design Decisions
Write-lock state management — state.lock prevents collision writes when multiple pipeline jobs run concurrently. All mutations are append-only to vault-events.log (JSONL). In-memory reads use a 30s TTL. → scripts/state-manager.js
Hybrid inference — Pipeline tries local Ollama first (zero cost, no rate limits). Falls back to cloud LLM if Ollama is unavailable. Scoring is deterministic via fixed seed. → scripts/evaluate_repo.py
Bot commits on heatmap — Git identity configured so automated commits register on the contribution graph. Pipeline runs as vybe-bot with a PAT scoped to repo only. → .github/workflows/harvester.yml
🚀 Quick Start
Prerequisites
Setup
# Clone
git clone https://github.com/sairaman436/vybe-intelligence-vault.git
cd vybe-intelligence-vault
# Pull required models
ollama pull nomic-embed-text # embeddings
ollama pull qwen2.5:14b # scoring
# Install dependencies
npm install
pip install -r requirements.txt
# Start everything (MCP server + orchestrator + web UI)
bash scripts/vault-init.sh
Verify
# Check service health, ports, and event log
bash scripts/vault-status.sh
Open http://localhost:3000 for the 3D intelligence map.
Configure Topics
Edit vault-core/config.yaml to control what gets harvested:
topics:
- ai-agents
- rag-architectures
- mcp-servers
- llm-inference
- next-gen-web
token_budget: 4096
score_threshold: 0.65
📊 Intelligence Analytics Dashboard
Real-time metrics generated from active vault contents.
🗄️ Core StorageResources tracked: 12,734 Active: 12,441 | Inactive: 293 |
📂 Archives & MapsArchive Files: 50,206 Builder Maps: 8 |
⚡ StatusTotal Vault Size: 62,940 files Last Update: 2026-07-18 04:18 IST Health: 🟢 Optimal |
📈 Trending Signals
Top rising resources based on momentum and community velocity.
- 🔼 Medium Members Can Listen To Any Medium Story With The Speechify Play Button | Speechify • Rank: +1
- 🔼 Home - The GitHub Blog • Rank: +1
- 🔼 LinkedIn • Rank: +1
- 🔼 GitHub Changelog • Rank: +1
- 🔼 The Agent Skills Directory • Rank: +1
🌟 New Discoveries
Fresh intelligence recently indexed into the vault.
- 🆕 EngineeringBringing more control over your connectorsJune 24, 2026By Mistral AI • Score:
0 - 🆕 langchain-ai/langgraph • Score:
0 - 🆕 ProductIntroducing Search ToolkitProduction search pipelines, anywhere.May 28, 2026By Mistral • Score:
0 - 🆕 CompanyAI Now Summit 2026Innovations for global enterprises solving the world’s hardest problems. May 28, 2026By Mistral • Score:
0
💤 Recently Inactive
Resources showing declined activity or relevance.
- None.
The stats shown here are generated from the current vault content. They refresh automatically when the bot finds changes.
📁 Repository Layout
vybe-intelligence-vault/
├── .github/
│ └── workflows/
│ ├── harvester.yml # Main 1h cron pipeline
│ └── rebuild-index.yml # Triggered on vault-core/ push
│
├── vault-core/
│ ├── config.yaml # Topics, token budgets, score thresholds
│ ├── vault-index.json # Compiled semantic node/edge graph
│ └── vault-events.log # Append-only JSONL event ledger
│
├── intelligence-map/ # React 19 + WebGL 3D dashboard
│
├── mcp-server/ # FastMCP integration server
│
├── scripts/
│ ├── evaluate_repo.py # Resource discovery + LLM scoring
│ ├── orchestrator/
│ │ └── context-injector.js # MCP context formatter
│ ├── state-manager.js # Lock-safe state writes
│ ├── build-index.js # Embedding + edge compiler
│ ├── vault-init.sh # Startup daemon (concurrent)
│ └── vault-status.sh # Port + health diagnostics
���
├── ai/ # Indexed resources by category
│ ├── agents/
│ ├── rag/
│ ├── models/
│ └── mcp/
│
└── search-index.md # Flat searchable index
🗺 Roadmap
- Vector search API over
vault-index.json(FastAPI endpoint) - Discord/Slack bot that answers "what's new in RAG this week?"
- Contributor scoring — track who surfaces the highest-value resources
- Export to Obsidian vault format
- GitHub App so others can run their own vault instance
🤝 Contributing
PRs welcome. Read CONTRIBUTING.md first.
# Run the scoring pipeline locally against a single URL
python scripts/evaluate_repo.py --url https://github.com/your/repo --dry-run
Bug reports → open an issue MIT License — see LICENSE

Comments