setup: complete wiki scaffold with schema, seed files, sample article, search script, and workflow docs

This commit is contained in:
truetype committed 2026-05-10 17:39:01 +02:00
1 parent 125f74c563
commit 6731ff4e97
5 files changed
+115 -1

No files matched your search

+51
View File
@@ -0,0 +1,51 @@
# Wiki Ingest Workflow
## How to Ingest a New Source
### Step 1: Add the source
Copy or save the document into the appropriate subdirectory of `raw/`:
- `raw/articles/` — web articles, blog posts
- `raw/papers/` — research papers, technical docs
- `raw/repos/` — code repository summaries
### Step 2: Trigger ingestion
Tell the LLM agent:
```
Read raw/articles/[filename] and ingest it into the wiki following AGENTS.md rules.
- Create or update wiki pages for key concepts, entities, and claims
- Add cross-references with [[wikilinks]]
- Update wiki/index.md
- Append to wiki/log.md
- Run the lint checklist
```
### Step 3: Review
- Check the LLM's summary of what it created/updated
- Browse the wiki in Obsidian or any markdown viewer
- Verify `wiki/index.md` is up to date
### Step 4: Commit
```bash
git add -A && git commit -m "ingest: [source filename]"
```
## How to Query
```
Using the wiki, answer: [your question]. Cite specific wiki pages.
```
## How to Lint
```
Run the full wiki lint per AGENTS.md checklist. Report issues and fix what you can.
```
## Naming Your Sources
Use clear, descriptive filenames:
- `raw/articles/2026-transformer-survey.md`
- `raw/papers/attention-is-all-you-need.pdf.md`
- `raw/repos/langchain-agent-notes.md`
## Tips
- Ingest one source at a time for best results
- Stay involved — read summaries, guide emphasis
- After 50+ sources, consider adding qmd for search
+40
View File
@@ -0,0 +1,40 @@
# LLM Wiki: A New Approach to Knowledge Management
Source: Andrej Karpathy's blog / GitHub Gist (2026)
## Summary
Andrej Karpathy introduced the concept of an "LLM Wiki" — a persistent, structured knowledge base built and maintained by an LLM agent. Unlike traditional RAG (Retrieval-Augmented Generation) systems that search raw documents at query time, an LLM Wiki is compiled once and kept current.
## Key Concepts
### The Problem with RAG
Traditional RAG retrieves from raw documents every time you ask a question. Results depend heavily on chunking strategy, embedding quality, and retrieval parameters. Good results are not retained.
### The LLM Wiki Approach
Instead of retrieving at query time, the LLM reads sources once and builds a structured wiki:
- **Raw sources** are stored in `raw/` (immutable)
- **Compiled knowledge** lives in `wiki/` as interlinked markdown pages
- **A schema file** (`AGENTS.md`) defines conventions and workflows
### Three Workflows
1. **Ingest**: Add source to `raw/`, LLM compiles into wiki (~10-15 pages per source)
2. **Query**: LLM reads `index.md`, finds relevant pages, synthesizes answer with citations
3. **Lint**: Periodic audit for contradictions, orphans, stale claims
### Compounding Knowledge
Good answers get filed back into the wiki as new pages. The wiki gets richer with every source added and every question asked. This creates a compounding knowledge effect.
## Technical Implementation
- Works with any LLM that can read/write files
- Tested with Claude Code, Codex CLI, Cursor
- Also works with local models via Ollama (Gemma, Llama, etc.)
- No special infrastructure needed — just a git repository of markdown files
- Optional: qmd for enhanced search at scale (100+ sources)
## Key Advantages
- No embedding models or vector databases needed
- Full version control via git
- Human-in-the-loop curation
- Knowledge compounds over time
- Works offline / on-premise
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# Wiki search utility — grep-based fallback (no qmd needed)
# Usage: ./scripts/wiki-search.sh "search query"
QUERY="$1"
if [ -z "$QUERY" ]; then
echo "Usage: wiki-search.sh <query>"
exit 1
fi
echo "=== Searching wiki for: $QUERY ==="
echo ""
grep -rl --include="*.md" -i "$QUERY" "$(dirname "$0")/../wiki/" 2>/dev/null | while read file; do
echo "--- $file ---"
grep -n -i "$QUERY" "$file" 2>/dev/null | head -5
echo ""
done
echo "=== Search complete ==="
+2 -1
View File
@@ -6,10 +6,11 @@
<!-- pages about people, organizations, tools -->
## Concepts
<!-- pages about ideas, techniques, theories -->
- [[llm-wiki-pattern]] — LLM Wiki methodology by Karpathy
## Sources
<!-- pages summarizing original documents, papers, articles -->
- [[sample-llm-wiki-intro]] — Introduction to LLM Wiki concept
## Comparisons
<!-- pages comparing tools, approaches, or ideas -->
+6
View File
@@ -2,5 +2,11 @@
> Append-only chronological record of all ingest and maintenance operations.
## 2026-05-10 17:30 — Wiki scaffold created
- Created: `AGENTS.md` (schema), `wiki/overview.md`, `wiki/index.md`, `wiki/topics/llm-wiki-pattern.md`
- Created: `raw/articles/sample-llm-wiki-intro.md` (sample article)
- Created: `scripts/wiki-search.sh` (grep-based search utility)
- Key takeaways: Initial setup following Karpathy's LLM Wiki pattern. Three-layer architecture in place (raw → wiki → schema).
---
*Log started: 2026-05-10*