LLM Wiki

← Docs hub

Architecture

llmwiki has two overlapping structures:

  1. The Karpathy three-layer wiki (conceptual): raw/ → wiki/ → site/
  2. The eight-layer build (implementation): how responsibilities are distributed across Python modules, HTML templates, scripts, CI, etc.

This document covers both.

Layer 1: Karpathy's three-layer wiki

From the original LLM Wiki gist:

raw/           IMMUTABLE source documents
    ↓          (llmwiki converts .jsonl → .md here)
wiki/          LLM-MAINTAINED pages
    ↓          (your coding agent writes here via /wiki-ingest)
site/          GENERATED static HTML
               (llmwiki builds here via `llmwiki build`)

raw/ — immutable layer

Everything under raw/ is treated as source-of-truth. The converter writes to it; nothing else should. If a source is wrong, fix the converter, not the output.

The converter writes one markdown file per session under raw/sessions/<project>/<date>-<slug>.md. Each file has YAML frontmatter (project, started, model, tools_used, gitBranch, etc.) and a Conversation body rendered turn-by-turn.

wiki/ — LLM-maintained layer

Your coding agent owns this layer entirely. It writes via the Ingest Workflow in CLAUDE.md:

wiki/
├── index.md          catalog of all pages, updated on every ingest
├── log.md            append-only chronological record
├── overview.md       living synthesis across all sources
├── sources/          one summary page per raw source (kebab-case slug)
├── entities/         people, projects, tools (TitleCase.md)
├── concepts/         ideas, frameworks, patterns (TitleCase.md)
└── syntheses/        saved query answers (kebab-case slug)

Pages interlink via [[wikilinks]]. Contradictions are recorded, not silently overwritten. Pages compound over time — every new source makes the wiki richer.

site/ — generated static layer

llmwiki build reads raw/ (and wiki/ if populated) and renders a complete static HTML site. Nothing here is hand-maintained. Safe to delete and regenerate any time.

Layer 2: The eight-layer build

Internally the code is organised into eight functional layers. Each layer has one clear responsibility, and each feature in docs/roadmap.md maps to exactly one layer.

┌──────────────────────────────────────────────────────┐
│  L7  CI / ops          .github/workflows/            │
│  L6  Adapters          llmwiki/adapters/             │
│  L5  Schema / docs     CLAUDE.md, AGENTS.md, docs/   │
│  L4  Distribution      setup.sh, .bat, .claude/      │
│  L3  Viewer            script.js in build.py         │
│  L2  Site              build.py (HTML + CSS)         │
│  L1  Wiki              CLAUDE.md workflows           │
│  L0  Raw               llmwiki/convert.py            │
└──────────────────────────────────────────────────────┘

L0 — Raw

Owner: llmwiki/convert.py

Reads .jsonl from the agent's session store (via an adapter), filters out noise records, runs redaction, normalises the output into markdown, and writes to raw/sessions/.

Key properties:

L1 — Wiki

Owner: your coding agent, following CLAUDE.md / AGENTS.md

llmwiki does NOT write to wiki/ directly. The agent does, via slash commands (/wiki-ingest, /wiki-query, /wiki-lint) that execute the workflows in the schema file.

L2 — Site (HTML generator)

Owner: llmwiki/build.py

Converts every file under raw/sessions/ (and any hand-authored files under wiki/) into static HTML. Uses python-markdown (the only runtime dep) — syntax highlighting runs in the browser via highlight.js loaded from a pinned jsdelivr CDN (v0.5, #73), so the build pipeline itself stays stdlib-only. Writes to site/.

Pages rendered (v0.9 surface):

L3 — Viewer (browser JS)

Owner: script.js (a string constant inside build.py)

Everything that happens in the browser, in vanilla JS:

Zero dependencies. No bundler. No framework. One file.

L4 — Distribution

Owner: the repo root + .claude-plugin/

How users install and run llmwiki:

L5 — Schema / docs

Owner: root-level markdown + docs/

Tells humans and agents how the system works:

L6 — Adapters

Owner: llmwiki/adapters/

One file per agent. Each subclass of BaseAdapter does three things:

  1. Knows where the agent writes its session store
  2. Walks that store to discover .jsonl files
  3. Derives a friendly project slug from the path

Everything else (record parsing, filtering, redaction, rendering) is shared in convert.py.

L7 — CI / ops

Owner: .github/workflows/ + tests/

Adding an adapter

See framework.md §5.25 Adapter Flow for the full contract. TL;DR: one new file at llmwiki/adapters/<agent>.py, one fixture, one snapshot test, one doc page, one README line, one CHANGELOG entry.

Design principles

  1. Stdlib first. Runtime dep: markdown only. Nothing else. Syntax highlighting runs client-side via a CDN-loaded highlight.js (v0.5, #73) so the build stays deterministic and offline-capable.
  2. Privacy by default. Redact everything sensitive before it hits disk.
  3. Idempotent everything. Re-running any command is safe and cheap.
  4. Localhost only. No network, no telemetry, no cloud. The user controls if/when to publish.
  5. One file per concern. build.py is one file, not a folder of templates. The whole HTML rendering lives there including CSS + JS.
  6. Agent-agnostic core. convert.py doesn't know which agent produced the .jsonl. Adapters translate.