OpenDocuments: A Lean, Self-Hosted RAG Tool That Just Works

Let’s be honest: you’ve probably tried at least three different RAG tools this year. You pasted your Notion docs into something, connected GitHub, maybe even wrestled with LangChain’s config hell—only to get vague answers, missing citations, or a Docker container that ate 8GB RAM and refused to start without a PhD in OpenTelemetry. Enter OpenDocuments — a lean, TypeScript-built, self-hosted RAG search tool that just works — and does it with fewer moving parts than a Raspberry Pi running Pi-hole.

At 55 GitHub stars (as of May 2024), it’s not trending on Hacker News yet. But I’ve been running it for 11 days — indexing 42 Notion pages, 17 GitHub repos (including private ones), and 3 shared Google Drive folders — and it’s become my daily driver for internal knowledge lookup. No vector database bloat. No forced cloud dependencies. Just ollama run llama3:8b + a single docker-compose up, and you’re querying your docs with inline citations.

Here’s why it stands out — and whether it’s ready for your stack.

What Is OpenDocuments? A RAG Tool That Doesn’t Overengineer

OpenDocuments is a minimalist, self-hosted RAG (Retrieval-Augmented Generation) frontend + backend that connects to your document sources (Notion, GitHub, Google Drive), chunks them intelligently, embeds them using your local LLM (via Ollama, OpenAI, or Anthropic), and serves answers with direct, clickable citations. It’s not another LLM wrapper with a fancy UI. It’s a tight, opinionated pipeline — built in TypeScript, MIT-licensed, ~2.4k lines of core logic — that assumes you want accuracy, traceability, and control — not bells.

Unlike LlamaIndex or privateGPT, OpenDocuments doesn’t ask you to write custom loaders or wrangle embeddings manually. It ships with first-class, authed connectors:

• Notion API v2 (OAuth2 flow, works with personal & team spaces)
• GitHub REST API (supports private repos, issue comments, PR diffs — yes, it indexes code and discussions)
• Google Drive API (scopes for https://www.googleapis.com/auth/drive.readonly, supports shared folders and .docx/.pdf/.txt/.md)

It uses Ollama’s embedding models (e.g., nomic-embed-text) by default — no Pinecone, no Weaviate, no ChromaDB config hell. Embeddings go straight into a lightweight SQLite DB (yes, SQLite — more on that later). Generation is handled by whatever LLM endpoint you point it to.

The UI is barebones React — no Tailwind bloat, no dark-mode toggle that breaks on Safari — just a search bar, ranked results, and a collapsible citation pane showing exactly which Notion page / GitHub PR / Drive doc contributed each sentence.

How to Install OpenDocuments: Docker-First, Zero-Config (Mostly)

I spun this up on a $5/month Hetzner CX11 (2 vCPU, 2GB RAM, 20GB SSD) — no Kubernetes, no reverse proxy at first. Just docker-compose.

Prerequisites

• Node 20+ (for local dev)
• Docker 24.0+
• Ollama 0.3.1+ (I’m using ollama run nomic-embed-text + llama3:8b)
• API tokens (Notion, GitHub, Google — see their respective docs for scopes)

Docker Compose Setup (v0.2.4 — latest as of May 2024)

# docker-compose.yml
version: '3.8'
services:
  opendocuments:
    image: joungminsung/opendocuments:0.2.4
    ports:
      - "3000:3000"
    environment:
      # Required
      OLLAMA_HOST: http://host.docker.internal:11434  # or your Ollama IP
      EMBEDDING_MODEL: nomic-embed-text
      LLM_MODEL: llama3:8b
      LLM_PROVIDER: ollama
      # Optional but recommended
      NOTION_INTEGRATION_TOKEN: "${NOTION_TOKEN}"
      GITHUB_TOKEN: "${GITHUB_TOKEN}"
      GOOGLE_CREDENTIALS: "${GOOGLE_CREDENTIALS_JSON_BASE64}"
      # SQLite persistence (volume required!)
    volumes:
      - ./data:/app/data
    restart: unless-stopped

⚠️ Critical note: host.docker.internal doesn’t work on Linux by default. If you’re on Ubuntu/Debian, add this to your docker-compose.yml under opendocuments:

extra_hosts:
  - "host.docker.internal:host-gateway"

Then run:

export NOTION_TOKEN=secret_xxxx
export GITHUB_TOKEN=ghp_xxxx
export GOOGLE_CREDENTIALS=$(cat credentials.json | base64 -w 0)
docker compose up -d

It boots in ~8 seconds. Hit http://localhost:3000, go to Settings → Sources, and toggle your integrations. No “setup wizard”. No “please wait while we index 10k docs”. It starts indexing as soon as you enable a source — and shows progress per document in the UI.

I indexed 1.2GB of mixed Notion/Drive content (142 docs) in 22 minutes on my CX11. RAM peaked at 1.4GB, CPU at 65% — sustained. Ollama (nomic-embed-text) used ~900MB alone. So yes — 2GB RAM is barely enough. I’d recommend 4GB minimum for anything beyond light personal use.

How It Compares to Alternatives You’ve Probably Tried

If you’re evaluating OpenDocuments, you’ve likely already battled with:

• Docq (self-hosted, Flask-based): Docq is solid, but requires setting up Postgres + Redis + Celery just to index a Notion space. Its UI is functional but dated. OpenDocuments trades extensibility for simplicity — no background workers, no async queues. It blocks on embedding, yes — but for <500 docs, that’s faster than managing 4 services.

• PrivateGPT (LLamaIndex + Chroma): PrivateGPT wants you to pip install 17 packages, tune chunk_size, manually handle PDF parsing, and pray your embeddings don’t drift. OpenDocuments auto-detects file types, uses nomic-embed-text’s built-in PDF/MD parsers, and stores raw chunks + metadata with document IDs — so citations point to exact headings, not just “page 3”.

• Jina AI Rerank + Weaviate: Overkill if you’re not doing enterprise-scale semantic search. Weaviate eats 3GB RAM before loading data. OpenDocuments + SQLite uses ~60MB on disk for the same 142-doc corpus. You can sqlite3 data/documents.db ".dump" and read the schema — it’s just documents, chunks, sources, embeddings.

• Notion AI (native): Convenient, but no cross-source search. Can’t ask “How does our GitHub PR template relate to Notion’s RFC process?” OpenDocuments can, because it flattens all sources into a single semantic space — and shows you which source each claim came from.

The kicker? OpenDocuments’ citation engine isn’t just “source: Notion page X”. It extracts heading context, line numbers for code, and document hierarchy — so if your GitHub PR description cites a Notion RFC, and you ask “Why was this change approved?”, it’ll quote both — with links.

Why Self-Host OpenDocuments? (Spoiler: It’s Not Just Privacy)

You can run this on a $5 VPS. You can run it on a Mac Mini in your closet. You can even run it on a Raspberry Pi 5 (I tried — it works, but embedding takes ~20s/doc; not recommended for >20 docs).

But self-hosting isn’t just about “keeping data local”. It’s about debuggability, version control, and iteration speed.

• git clone https://github.com/joungminsung/OpenDocuments && npm run dev boots a hot-reloading dev server in 4 seconds. You can console.log() inside the Notion loader, tweak chunking logic (it uses markdown-it + custom heading-aware splitting), and test immediately.

• The config is environment-based — no YAML files to misplace. Want to switch from Ollama to anthropic/claude-3-haiku-20240307? Just change LLM_PROVIDER=anthropic and ANTHROPIC_API_KEY=xxx. No code changes.

• It logs everything to stdout — including embedding latency, chunk count per doc, and LLM token usage. No black-box telemetry. No “why is this slow?” mystery.

Also: no vendor lock-in on embeddings. Unlike tools that bake in OpenAI’s text-embedding-3-small, OpenDocuments uses Ollama’s nomic-embed-text by default — but you can swap in jina-embeddings-v2-base-en or even a fine-tuned local model — just change the EMBEDDING_MODEL env var.

System Requirements & Real-World Resource Usage

Let’s cut the fluff.

I ran htop while indexing 84 GitHub issues + PRs (32MB raw):

• opendocuments process: avg 320MB RAM, 45% CPU
• ollama serve: 890MB RAM, 95% CPU (embedding burst)
• sqlite3 (via Node): <15MB

No swap thrashing. No OOM kills. But if your VPS has only 2GB RAM and you’re also running PostgreSQL + Nextcloud, don’t do it. This isn’t lightweight like filebrowser — it’s a real ML pipeline, just stripped of the enterprise scaffolding.

Also: it does not support real-time sync yet. You must manually click “Re-index” in Settings (or hit /api/v1/sync via curl) to pull fresh Notion/GitHub changes. There’s no webhook support — yet. (The GitHub issue #42 is open; author says “soon”.)

Is OpenDocuments Worth Deploying? My Honest Take

Yes — if you value simplicity, transparency, and cited answers over flashy dashboards or SaaS-grade uptime.

I switched from Docq after 3 days of debugging Redis connection timeouts. OpenDocuments just… worked. My team uses it to search RFCs, PR templates, and internal runbooks — and for the first time, we trust the citations. If the answer says “See Notion page ‘API Rate Limits’ — section ‘Burst Handling’”, we click and see it. No hallucination. No “based on our docs”.

Rough edges? Absolutely.

• No multi-tenancy: It’s single-user. No role-based access, no workspace isolation. Fine for a team of 5, not for 50.
• No PDF OCR: Upload a scanned PDF? It’ll skip it. Only text-based PDFs (like those generated from LaTeX) work. (PR #61 is open.)
• UI is functional, not beautiful: No search history, no saved queries, no keyboard shortcuts (/ to focus search doesn’t work — yet).
• No caching layer: Every query hits Ollama fresh. No Redis cache for repeated questions. You will see latency spikes if your LLM is busy.

But — and this is key — none of these are architecture problems. They’re all solvable with <100 lines of TypeScript. The codebase is designed to be hacked on. The src/connectors/ folder is clean. The src/embedding/ logic is 3 files. I’ve already patched in local file system indexing (not in upstream yet) — took me 45 minutes.

So is it production-ready? For a startup’s internal wiki? Yes. For a regulated financial team needing audit logs? Not yet — but the foundation is there.

The TL;DR:
✅ Deploy if you want a lightweight, self-contained, citable RAG tool — and you’re comfortable with Docker + env vars.
❌ Skip if you need SSO, RBAC, OCR, or HA clustering out of the box.

I’m keeping it. Not as a “maybe”, but as my default document search — because in 11 days, it hasn’t lied to me once. And in AI tooling? That’s rarer than a bug-free Dockerfile.

Want the patched version with local file support? I’ll drop the diff in the comments — or ping me on Mastodon @[email protected].