Combining Agent Memory, Information Retrieval, and Human Knowledge Base - Karparthy's LLM Wiki

[ladenaw]

Wikis and similar solutions (e.g., Loop, Confluence, Notion, ...) are great for organizing and synthesizing knowledge from raw sources like papers, articles, and websites. Using them, you can create articles, cross-link, and categorize them. You may also reflect on raw sources and add your own thoughts. The resulting knowledge bases are unique; they are tailored to your personal or team's use cases, projects, and research needs. When searching them, you will find what you need fast and in the format you require.

Though personal wikis are great, maintaining them may become cumbersome as they grow. They aren't static snapshots but evolving entities. Additional sources get ingested, existing information becomes outdated, and your needs and use cases might change over time. All of this requires the knowledge base to grow, forget, evolve, and restructure. This is especially true for exploratory work, where the best form of knowledge organization is unknown in advance. Talking from personal experience, this is when we often consider starting from scratch or abandoning knowledge bases altogether. 

Andrej Karpathy's LLM Wiki concept promises to solve this problem. It offloads the tedious, manual parts of building and maintaining knowledge bases to an LLM. Instead of painfully keeping sources up to date, checking links, refactoring articles, and updating conclusions manually, you supply raw sources, define purpose, and research flow, and let the LLM take care of the rest. 

Wikis are great, but maintaining large wikis takes a lot of effort.
LLM Wiki offloads the tedious bookkeeping tasks to AI agents.

In this article, we introduce you to the LLM Wiki concept, help you get started with an LLM Wiki, highlight its merits, and discuss open questions of the approach. If you are already familiar with the concept, you may want to skip the introductory section and read about two promising business use cases of the LLM Wiki as well as its relation to information retrieval, agentic memory, and human wikis.

The Idea

Andrej Karpathy proposed the LLM Wiki concept in April 2026. He describes a way to build and consume knowledge bases aided by modern agent harnesses such as Claude Code.

The core setup is very simple: just instruct your favorite agent harness to set up a knowledge base based on the original idea in collaboration with you.

You tell it that the goal is to have a setup consisting of a folder with raw source files (e.g., PDFs, markdown files, etc.) and a wiki folder that shall be maintained automatically. Additionally, a schema file with further instructions (e.g., CLAUDE.md, AGENT.md) regarding metadata, lifecycle processes, logging, and indexing is required. Please take a look at an example for a CLAUDE.md file attached below the blog post. Copy the file's content to a CLAUDE.md in a directory of your choice and start your LLM-maintained wiki!

Schema File Excerpt

# LLM Wiki Schema

This repo is an LLM-maintained wiki. The LLM incrementally builds and maintains a structured, interlinked collection of markdown pages from raw source documents. The human curates sources and directs the analysis. The LLM does the summarizing, cross-referencing, filing, and bookkeeping.

## Architecture

Three layers:
[...]

The schema file is the soul and core of the whole setup – in its most basic form, the LLM Wiki concept works solely without code. It describes the agent's wiki schema, rules, and operations. This is the contract the agent follows; it defines the wiki. The original LLM Wiki concept contains three crucial operations:

• Ingest: The user adds new files to the raw folder, and the agent ingests them into the wiki.
• Query: Information extraction from the wiki by asking questions to the agent.
• Lint: The agent checks the wiki for shortcomings such as inconsistencies, gaps, missing links, etc., and updates it if necessary.
  
  

In addition, two additional concepts make the LLM Wiki auditable and easy to search:

• index.md: As part of the wiki, the agent maintains an index.md file that links to all the wiki pages and, if applicable, clusters them by categories. This allows for easy navigation of the wiki content for all three operations.
• log.md: Every time the wiki is updated, the agent adds an entry to the log.md file. This is an append-only, chronological record of wiki changes. Across multiple agent sessions, this helps the agent to understand what it did before.
  
  

Basic LLM Wiki Architecture

The simple setup and prompt-based nature of the concept invite you to test your own implementations and try out customizations. Consequently,  the LLM Wiki idea sparked a lot of interest, discussion, and personal interpretations (see the comments under the original post). Some integrate their LLM Wikis with human interfaces, such as Obsidian. Others optimize wiki operations, providing open-source implementations that integrate technologies such as vector database similarity search or that provide use-case-specific variations of the original idea. The core concept, however, remains the same: use the power of LLMs to distill, synthesize, connect, and understand knowledge while always adapting to the user's evolving needs.

 

Three Facets of LLM Wiki

LLM Wiki was originally positioned as a personal knowledge base. Nevertheless, we find that it can potentially take on three different functions in agent-based software systems:

Information Retrieval 
LLM Wiki serves as the rich index of an information retrieval pipeline. In this capacity, it opts to increase retrieval efficiency in two ways. First, it may surface information that made it to the index (high efficiency, like a cache hit). Second, it links to full sources for further investigation (lower efficiency) and captures the original information context (e.g., full docs or chunks). Keeping these two capabilities in mind, LLM Wiki closely resembles known layered, progressive-disclosure, and graph-based RAG approaches as well as enriched and hierarchical indexing strategies. While simple information retrieval operates in zero-shot mode, existing (agentic) retrieval approaches can research sources for answers to multi-hop problems, as can LLM Wiki by binding sources across wiki pages and subsequent agentic read steps.

Agent(ic) Memory  & Context Engineering (see our prior blog posts for reference)
From this perspective, LLM Wiki serves as an agent’s self-maintained, token-level knowledge store. In this capacity, it is built and also predominantly consumed by an agent, not a human. In contrast to pure information retrieval, it aims to distill higher-order knowledge from noisy, overlapping, potentially contradictory, volatile, and incomplete information sources. Knowledge distillation and summarization assume an understanding of expected future utility to improve the agent’s performance and are thus opinionated. Both require up-front reasoning and incur token costs that may be recouped over time. Summarization, as a lossy compression, supports context engineering by helping retain attention and reduce the number of context tokens. Building agentic memory based on files is a known and applied strategy that leverages the file-based nature of widespread agent harnesses like Claude Code.

Human Wiki/Knowledge Base 
Reflected in the original "wiki" framing of the approach is the desire to build and maintain a persistent, compounding artifact for human consumption. A wiki inherently is a "human-in-the-loop" approach. It is a knowledge base that a user actively monitors, both for absolute content and for ingestion diffs (basically learning bits to update the user’s mental model). This highlights the potential of an LLM Wiki as a human-facing asset. It provides familiar cross-linking structures, article lengths, and formatting, which enable easy onboarding and convenient consumption. It also supports manual edits and lets users spot blanks in the knowledge base and assess its overall structure and extent.

Where LLM Wiki Can Help Business Use Cases

Looking at the three potential merits of LLM Wiki, you may be wondering, "Is this the magical one-size-fits-all solution?" or "Is it just a clunky compromise between more focused solutions?" While we cannot answer these questions conclusively, we want to share our thoughts on two business use cases that may benefit from LLM-Wiki's properties:

Use Case Example: Ticketing and Customer Support Systems
Software and customer support tickets often require human experts to hold a lot in mind simultaneously: knowledge of the codebase or product, common error patterns, resolution paths, and communication best practices. This complexity – combined with recurring issues and the constant onboarding of new team members – makes support a strong candidate for an LLM Wiki approach.

The raw sources are historic and ongoing ticket interactions, existing documentation, and code bases. Tickets are partly overlapping, potentially outdated, implicit, and noisy. Yet they contain exactly the kind of higher-order knowledge – recurring failure modes, proven resolution paths, escalation patterns – that an LLM Wiki is built to distill. The wiki surfaces this latent knowledge in a structured, human-readable form that supports agents and human operators alike in using and editing it.

LLM Wiki is particularly well-suited here because the knowledge base must stay auditable and editable. Support policies, communication tone, and resolution strategies change over time, requiring active human oversight. The wiki's
transparent, file-based structure makes it straightforward to correct errors, realign focus, and track what the agent knows. This keeps human operators knowledgeable, in control, and ready to stay operative should AI assistance fail. Finally, repeating errors justify an upfront token investment to build and maintain the wiki.

Use Case Example: Code Documentation

Another potential enterprise use case is automatic code documentation by and for agents. Code files are the raw source files that the agent ingests and processes. The resulting wiki pages serve as documentation that can be used by coding agents and humans alike.

This is a good fit for LLM Wiki because code files contain raw information, are hard to read, and do not easily convey higher-level concepts, architecture, and knowledge. LLM Wiki bridges this gap by front-loading the reasoning on top of the code. Once a human or agent wants to work with the code, the knowledge is already there. This also connects to our previous blog post on orchestrating autonomous coding agents. There are multiple ways to integrate autonomous coding agents as LLM Wiki agents into the software development lifecycle:

• Update documentation regularly on a schedule (e.g., every week).
• Update documentation whenever the codebase changes (commit the default branch).
• Update documentation for every release (e.g., tag in Git).
  

Depending on who consumes the documentation and how it’s used, the schema and resulting documentation will differ. LLM Wiki excels in this adaptability.

Efficiency, Stability, and Open Questions

Just like agentic memory, LLM Wiki relies on lifecycle strategies for content formation, evolution, and retrieval. Most implementations of LLM Wiki use a proprietary agent harness, such as Claude Code, for these purposes. This caters to the harness’s ability to manage and reason on a file-based wiki and is reported to work well at a moderate scale. Still, it remains unclear how this approach fares in practical applications and at bigger scales:

• What are the scalability limits and cost-benefit ratios across these three main purposes, and how do we measure them?
• Under which business conditions, and to what extent, are upfront investments in knowledge base construction justified by their ability to either outperform or complement pure retrieval approaches?
• Can the retrieval-side performance of the vanilla LLM Wiki be increased by optimized interfaces (vector-embedding of wiki and/or backlinks, indexing of wiki, graph algorithms on top)?
• How stable is the approach with respect to input material, prompting, model choice, ingestion strategies (e.g., batching), and how much human intervention is needed to guide a successful and economic build-up?
• How can this approach evolve to also work reliably on multimodal input material (videos, pictures, audio, hybrid)?

Feel free to reach out and share your thoughts on these open questions, as well as any experience you may have with the approach! 

# CTO Research and Innovation Munich

@ladenaw 

@dDaniel

CLAUDE.txt