The corpus,
readable by your AI.

The setup is unfashionable. The corpus is plain markdown in a git repository. A build job tokenizes it into a BM25 index, which is a JSON blob the API loads into memory on cold start. There's no vector store and no retrieval pipeline. When a search call comes in, BM25 picks the candidates; the model that asked does the rest of the thinking.

That choice came from reading Karpathy on what he calls the LLM Wiki pattern.

"Treat the LLM as a compiler that reads raw source documents and produces a structured, interlinked wiki. Don't rely on complex RAG pipelines or vector databases."

His point: an embedding is opaque. You can't read it, and when retrieval surfaces the wrong page you can't really say why. A markdown file isn't opaque. If an answer cites /glossary/msp.md, you open the file and see exactly what the model was reading.

The scale rule he gives is roughly: under a few hundred pages, an llms.txt index by itself is enough. Past that, add BM25. Only reach for vectors when BM25 starts missing things. The corpus is around 1,150 documents, so I'm at the BM25 step, and I haven't hit a recall problem yet.

A pattern, applied twice: falsafa.ai

I built falsafa.ai (source) before this. Same setup, applied to classical and philosophical works: markdown as the source, three small read tools (search_corpus, read_chapter, get_passage), a synthesised wiki layer for navigation, and citation handles that take you back to the original passage. Liberty Lighthouse is that pattern run again on a different body of text.

Repeating the pattern is the point. The texts don't transfer. Philosophy and Indian policy aren't comparable subjects. But the shape of the work is the same: bounded bodies of text with internal references and contested readings, where an agent that wants to engage seriously has to cite a specific passage rather than gesture at the corpus.

A lot of what's here is salvaged from the falsafa.ai work: the citation block on every response, the rule that every page has a markdown sibling, the choice not to host any inference on the server. If you want to build something similar, the source is on GitHub and the schema is at /AGENTS.md.

What the agent gets back

Every response carries a citation block like this:

{
  "canonical_url": "https://liberty-lighthouse.vercel.app/glossary/msp/",
  "markdown_url":  "https://liberty-lighthouse.vercel.app/glossary/msp.md",
  "title":         "MSP",
  "kind":          "glossary",
  "last_modified": "2026-04-26"
}

An agent should quote the answer and cite canonical_url so the user lands on the human page. The markdown_url is for traversal to the next document, kind labels the shape (faq, glossary, video, topic, wiki, external_post, external_book_chapter), and last_modified exists so the agent can decide whether a cached answer is still current. Those five fields are the entire contract.

The corpus is openly classical-liberal. On contested policy questions, an agent summarising it should say so rather than present the position as neutral consensus. The citation block helps with that, since every paraphrase points at a specific file you can open and verify against.