The Open Knowledge Format (OKF) is an open, vendor‑neutral specification for representing the metadata, context, and curated knowledge that AI systems — especially agentic ones — need in order to produce accurate, actionable results. It formalises the LLM wiki pattern (popularised by Andrej Karpathy) into a portable, interoperable standard: a directory of markdown files with YAML frontmatter and a small set of agreed‑upon conventions, readable and writable by both humans and AI agents without proprietary tools.
OKF v0.1 requires no compression scheme, runtime, or SDK. A bundle is just markdown (readable anywhere, renderable on GitHub), just files (tarball‑able, git‑hostable, mountable), and just YAML frontmatter for the handful of fields that need to be queryable. It is designed to solve the fragmentation of internal knowledge in organisations — schemas, metrics, runbooks, API deprecations, join paths, and other “atoms of knowledge” currently scattered across wikis, catalogs, code comments, shared drives, notebooks, and engineers’ heads.
Key Points
The problem is fragmentation, not tooling. Organizational knowledge needed by models (table schemas, metric definitions, runbooks, join paths, deprecation notices) is scattered across metadata catalogs, wikis, shared drives, code comments, docstrings, notebooks, and engineers' heads. Every agent builder re-solves context assembly; every catalog vendor reinvents the same data model; knowledge stays locked behind the surface that created it.
The answer is a format, not another service. A useful knowledge representation must be producible without an SDK, consumable without an integration, portable across systems and organizations, version-controlled alongside the code it describes, and simultaneously human-readable and agent-parseable with no translation layer.
OKF bundles are directories of markdown files (.md), each representing one concept (e.g. a table, dataset, metric, playbook, runbook, or API), with the file path serving as the concept's identity.
A small set of YAML frontmatter fields (type, title, description, resource, tags, ) provides structured queryability; only is required.
timestamp
type
Knowledge as a living wiki. Rather than having models re-search the same documents for the same facts, teams give agents a shared markdown library that improves over time: agents do the bookkeeping (reading, updating, cross-referencing many files in one pass), while humans curate content and manage it like code.
The format lives in version control and requires no SDK, runtime, or special integration.
OKF follows three design principles: minimally opinionated, producer/consumer independence, and “format, not platform”.
Reference implementations include a BigQuery enrichment agent, a static HTML visualiser, and three sample bundles for GA4, Stack Overflow, and Bitcoin datasets.
Concepts
LLM wiki pattern — a practice (popularised by Andrej Karpathy) of using a shared markdown wiki that an AI agent can consult and update, turning knowledge maintenance into a task the LLM handles while humans curate.
Bundle — the top‑level directory of markdown files representing concepts; the unit of distribution. May optionally contain reserved filenames (see below).
Concept document — a single markdown file in an OKF bundle, with YAML frontmatter and a body that can contain prose, links, code, or diagrams. One concept per file; the file path is the concept's identity. A concept can be anything captured as knowledge: a table, dataset, metric, playbook, runbook, or API.
Frontmatter fields — the small structured set: type, title, description, resource, tags, timestamp. Only type is mandatory.
Reserved filenames — index.md for progressive disclosure as agents navigate the hierarchy, and log.md for chronological change history.
Cross-links — ordinary markdown links between concepts, making the directory a graph richer than the parent/child relations implied by the filesystem.
Producer / consumer independence — OKF cleanly separates knowledge creation from consumption; a hand‑authored bundle works with any agent, a machine‑generated bundle can be browsed in any viewer, and a bundle synthesised by one LLM can be queried by another.
Reference implementations — an enrichment agent (BigQuery → OKF), a static HTML graph visualiser, and three sample bundles included in the repository.
Related convergences — Obsidian vaults wired to coding agents, the AGENTS.md / CLAUDE.md convention files, index.md/log.md artifact repos, and “metadata as code” repositories in data teams.
Details
The problem: fragmented context
Modern AI agents need relevant internal knowledge — table schemas, business metric definitions, incident runbooks, join paths, deprecation notices — to produce accurate results. This knowledge lives in siloed systems: metadata catalogs, wikis, code comments, shared drives, docstrings, notebooks, and the heads of a few senior engineers. Every agent builder solves the same context‑assembly problem from scratch, and each vendor reinvents its own data model. OKF aims to provide a common interchange format that any system can produce and any agent can consume.
The pattern: Knowledge as a living wiki
The LLM wiki pattern treats knowledge as a shared markdown library that grows over time. Developers curate the content, while AI agents handle the drudgery of reading and updating cross‑references. This pattern has reappeared in many forms (Obsidian vaults, AGENTS.md convention files, “metadata as code” repositories), but each implementation is bespoke. OKF formalises the conventions needed for interoperability.
How OKF works
An OKF bundle is a directory of markdown files. Each file is one concept (table, metric, runbook, API, etc.). The file path serves as the concept’s identity. Structured, queryable information lives in a small YAML frontmatter block at the top; everything else lives in the markdown body:
---
type: table
title: Orders
description: Raw order events from the web store
resource: bigquery://project.dataset.orders
tags: [ecommerce, raw, event]
timestamp: 2025-03-15T10:30:00Z
---
The body of the file is free‑form markdown. Concepts link to each other with standard markdown links, turning the directory into a rich graph. Bundles may include index.md (for hierarchical progressive disclosure) and log.md (for chronological change history). The full v0.1 specification — conformance criteria, cross‑linking rules, reserved filenames — fits on a single page.
The shape will be familiar to anyone who has used Obsidian, Notion, Hugo, or one of the LLM wiki patterns; OKF's contribution is formalizing the conventions needed to make those bespoke patterns interoperable.
Rendering diagram…
Three design principles
Minimally opinionated — Only the type field is required. All other fields, types, body sections, and content models are left to the producer. The spec defines the interoperability surface, not the content model.
Producer/consumer independence — A bundle written by a human can be consumed by an AI agent; a bundle generated by a pipeline can be browsed in a visualiser; a bundle synthesised by one LLM can be queried by another. Who writes knowledge and who reads it are cleanly separated; the format is the contract, and tooling at either end is independently swappable.
Format, not platform — OKF is not tied to any cloud, database, model provider, or agent framework. It will never require a proprietary account or SDK. The value comes from how many parties speak the format, not from who owns it.
What was shipped with v0.1
Reference agent — walks a BigQuery dataset, drafts an OKF concept document for every table and view, then runs a second LLM pass crawling authoritative documentation to enrich each concept with citations, schemas, and join paths.
Static HTML visualiser — turns any OKF bundle into an interactive graph view in a single self‑contained file; no backend, no install, no data leaves the page.
Three sample bundles — conformant OKF bundles for GA4 e‑commerce, Stack Overflow, and Bitcoin public datasets, produced by the reference agent and committed to the repository as living conformant examples.
These are deliberately proofs of concept: nothing in the format requires a particular agent framework, LLM, HTML, or graph view. Producers and consumers are expected to proliferate well beyond them.
Status, adoption, and next steps
OKF v0.1 is an open, versioned starting point designed for backward‑compatible growth rather than a finished standard, published openly from day one to encourage contributions, alternative implementations, and adoption beyond Google. Google Cloud’s Knowledge Catalog has been updated to ingest OKF and serve it to its agents. The repo, spec, and sample bundles are hosted at the GitHub repository.
The format itself is the primary contribution — the tools lower the cost of trying it out, but OKF is explicitly open to contributions and adoption beyond Google products. The community is encouraged to:
Read the spec and write producers for source systems (databases, documentation sites, catalogs).
Write consumers (viewers, search indexes, agents that reason over bundles).
Try the reference implementations against their own data.
File issues, send PRs, or propose extensions via the GitHub repository.