Architecture

Status: built. Every milestone below is implemented; the package layout reflects the code.

The problem CoDNA solves

A company's knowledge is scattered across Slack threads, Google Docs, PDFs, and people's heads. Teams increasingly ask an AI agent instead of searching, but the agent has nowhere trustworthy to look — and if you give it access to everything, you have quietly destroyed your access controls.

CoDNA is a knowledge base whose primary consumer is an AI agent, and whose primary constraint is that the agent must see exactly what the person it is acting for is allowed to see. Not more.

Shape of the system

┌──────────────────────────────┐        ┌────────────────────────┐
│  Claude · ChatGPT · Gemini   │        │   Web UI (humans)      │
│  any MCP client              │        │   Svelte, embedded     │
└──────────────┬───────────────┘        └───────────┬────────────┘
               │ MCP (streamable HTTP)              │ REST + cookies
               ▼                                    ▼
    ┌───────────────────────────────────────────────────────────┐
    │                  codna — one static binary                │
    │                                                           │
    │  ┌─────────────────────────────────────────────────────┐  │
    │  │ transport: http handlers · mcp tools · embedded UI   │  │
    │  │ Both routes resolve to the same Actor.               │  │
    │  └──────────────────────┬──────────────────────────────┘  │
    │                         ▼                                 │
    │  ┌─────────────────────────────────────────────────────┐  │
    │  │ domain: documents · revisions · linkgraph · access  │  │
    │  │ suggestions · automations · ingestion · search      │  │
    │  └──────────┬──────────────────────────────┬───────────┘  │
    │             │                              │              │
    │  ┌──────────▼───────────┐     ┌────────────▼───────────┐  │
    │  │ outbox worker        │     │ scheduler (cron)       │  │
    │  │ goroutine pool       │     │ FOR UPDATE SKIP LOCKED │  │
    │  └──────────┬───────────┘     └────────────┬───────────┘  │
    └─────────────┼──────────────────────────────┼──────────────┘
                  │                              │
          ┌───────▼───────┐              ┌───────▼──────────┐
          │  PostgreSQL   │─────────────▶│    Typesense     │
          │  source of    │  via outbox  │  derived index   │
          │  truth        │              │  (never truth)   │
          └───────────────┘              └──────────────────┘

Three rules hold this together:

1. One permission check. The domain layer decides who can do what. The REST API and the MCP server are transport, not policy. A rule fixed once is fixed everywhere. Nobody is exempt from it: there are no administrators, and management — which runs the instance — can read nothing (adr/0013).
2. Postgres is truth; everything else is derived. Typesense can be dropped and rebuilt from Postgres at any time. It is never written to directly.
3. One process, one binary. API, MCP, workers, scheduler, and UI ship together. There is no separate queue, no sidecar, and no node_modules at runtime.

The event spine

Every mutation writes its change and an event row in the same transaction:

BEGIN
  UPDATE documents          …
  INSERT document_revisions …
  INSERT events             …   ← same transaction
COMMIT

A worker then drains events and fans out. This is the transactional outbox pattern, and it is the single most load-bearing decision in the design:

Consumer	What it does
Search indexer	Upserts or removes the document in Typesense
Link resolver	Re-extracts links, resolves pending ones, marks stale and broken edges
Automation runner	Matches enabled automations against the event and runs their scripts
Notifier	Raises access, broken-link, and suggestion notifications
(seam) AI workers	Not built. Server-side distillation plugs in here later as one more consumer.

Why it matters. Without the outbox, a write to Postgres followed by a write to Typesense can half-succeed: the document exists but is unsearchable, or worse, a revoked share is still visible in search results. With it, the event is committed atomically with the change, so a consumer that crashes simply retries — the index converges, it never silently diverges.

It also makes consumers pluggable. The hybrid AI position — agents drive today, server-side workers maybe later — costs nothing precisely because "later" means adding a consumer, not restructuring the write path.

Consumers must be idempotent: delivery is at-least-once, not exactly-once.

A failed event is tried again after a wait that doubles from half a minute up to an hour, ten attempts in all -- about three hours, enough to ride out Typesense or the database being away -- before it becomes a dead letter for a person to look at. The event remembers which consumers already succeeded, and a retry runs only the rest, so a failing indexer does not make the notifier tell anyone twice. A consumer that panics fails its event like any other error, and the process stays up.

Dispatch uses Postgres LISTEN/NOTIFY through pgx to wake the worker immediately, with a periodic poll as a floor so a missed notification delays work rather than losing it.

Claiming an event takes two mechanisms, not one. FOR UPDATE SKIP LOCKED stops two workers racing on the same row during the claiming statement; a claimed_at lease stops the second worker picking it up again after that statement commits, which it otherwise would, since the row stays unprocessed until the consumer finishes. An expired lease returns the event to the pool, so a worker that dies mid-flight delays its events rather than stranding them.

The lease suits work measured in seconds. Reading an imported export can take many minutes, so imports have a worker of their own beside the outbox's, claiming with the same two mechanisms but holding its lease by renewing it as it goes (ingestion.md). The pages it writes are ordinary events on the outbox, so the index, the link graph and folder shares hear of each one as they would of any other.

Live updates

A page someone has open hears when what they can see, or what waits on them, changes, and fetches it again: share a document with Dana and her open tab shows it within a second — the inbox badge, Home, the sidebar's open folders, the Inbox, a folder's page — without a reload. Taking a share away empties them the same way.

document_access ─┐ triggers   pg_notify('codna_live',     ┌─ hub (internal/live) ─┐  GET /api/me/live
notifications   ─┼──────────▶ 'access group:<id>',  ─────▶│ one LISTEN connection │────────────────▶ page
link_suggestions ┘            'inbox user:<id>')          │ per process           │  event: changed
                                                          └───────────────────────┘  {"what":"access"}

• Postgres says what changed and for whom, from row triggers (migration 00007), so every way a share or a notification is made — the API, MCP, automations, the command line, a quarantine — is heard without any of them having to remember to. A bulk share signals each principal once: Postgres drops a notification identical to one already sent in the same transaction.
• The hub holds one connection of its own, outside the pool, and wakes the streams of the people a signal is for: a group's members, looked up then, or every stream for Everyone, which has no member rows. A stream folds whatever arrives before it writes into one event per kind.
• The stream says only that something changed, access or inbox, never what. The page asks the API again as its person, and the API decides what they may see, so nothing here can leak a document. It holds no database connection; it ends after fifteen minutes and the browser reconnects, which checks the session again.
• The page in view is watched on the same stream (?watch=<document id>, migration 00018): a new version saved, a deletion or restore, or a quarantine wakes only the streams watching that document, and each looks it up again as its person and says which version is current — event: document, {"id":…,"version":N} — or that it is gone for them. After gone it says nothing more of it, so a page someone may not read cannot be watched for when it changes.
• Nothing is queued, and nothing needs to be. A signal missed — the hub reconnecting, a page between streams — costs a refresh a little later: the hub tells everyone to look again when it listens again, and a page looks again whenever its stream reopens or the tab comes back into view. A hidden tab lets go of its stream, so background tabs hold no connections.

A page being read shows the new version. A page being written merges it into the text, in the browser (web/src/lib/editor/merge.ts): both sides are measured from the version the edit began from, line by line and then, for lines both changed, word by word. What only one side changed is taken from that side, placed in the editor where it belongs so the cursor keeps its place, and kept out of the undo history so undoing one's own typing never takes out someone else's save. Changes that overlap or touch are not merged: the writer is told, before they press Done, and compares in the conflict panel. A save refused as stale is merged the same way once and sent again, and a draft left on the device from an older version is merged with what was saved since. Nothing is ever saved over someone else's version without either merging it or the writer choosing to.

The stream is server-sent events, and a proxy must not buffer it. CoDNA sends X-Accel-Buffering: no, which turns buffering off in nginx, and a comment every 25 seconds, inside nginx's default 60-second read timeout; Caddy passes event streams through as they are.

Automations: events and schedules

Automations consume the same event stream, plus a clock. They are user-authored JavaScript, written in practice by an AI agent on request.

events ──────────▶ ┌──────────────────┐
                   │ automation       │──▶ QuickJS in wazero ──▶ proposed actions
scheduler ───────▶ │ matcher          │                              │
  (cron, Postgres) └──────────────────┘                              ▼
                                                    host validates each action against
                                                    the automation OWNER's permissions
                                                                     │
                                                                     ▼
                                                      applied in one transaction

Triggers fall into four families — document lifecycle, access changes, link-graph changes, and schedule; an ingested document arrives as document.created like any other. A scheduled automation has no subject document, which is why the capability API includes bounded reads (codna.find, codna.get) and not only actions. Those reads run with the automation owner's visibility, so a script cannot read past its author.

The scheduler is Postgres-backed rather than in-process: due automations are claimed with FOR UPDATE SKIP LOCKED, so running several replicas neither double-fires nor drops a tick.

Full contract in automations.md.

Where the AI lives

CoDNA does not call an LLM. Agents connect over MCP and do the thinking:

person: "build a wiki page from the onboarding PDFs I uploaded"
agent:   search(space=raw, query="onboarding")
         get_document(...) ×4
         create_document(space=wiki, ...)
         suggest_links(...)   → person approves

This means no provider API key is required to run CoDNA, no inference cost, and no opinion about which model is best. Every MCP-capable client works on day one.

The seam for server-side workers exists (above) for the cases agents cannot cover: bulk backfill of a 10,000-document import, or scheduled re-distillation. That is deliberately not in the first milestones.

Package layout

cmd/codna/              serve · migrate · bootstrap · reindex · ingest · import · export ·
                        people · groups · tokens · status · events (running the instance)
internal/
  domain/               core types and the permission policy — pure, no I/O
  store/                sqlc-generated queries; hand-written SQL lives in db/queries/
  search/               Typesense collections, indexing, ACL-filtered queries
  indexer/              the outbox consumer that keeps search in step with Postgres
  markdown/             goldmark parsing, frontmatter, link extraction and editing
  document/             create, update, move, delete, revisions, diffs, backlinks
  linkgraph/            extraction, resolution, state transitions and their events
  sharing/              grants, revokes, share suggestions, access events
  people/               people, groups, and handing on what someone owned; management's to change
  notify/               notifications and link decisions
  automation/           registration, runner, outbox consumer, scheduler
  sandbox/              QuickJS on wazero: limits, host calls, the JS prelude
  outbox/               event worker, consumer dispatch, cascade depth
  live/                 telling open pages what changed for their person: LISTEN, fan-out
  maintenance/          retention jobs, under an advisory lock
  status/               operational counts and dead-letter handling, for management
  audit/                the management audit: what management did, append-only
  breakglass/           finding, unsharing and quarantining documents by metadata, never content
  ingest/               file conversion, raw document creation
  storage/              blob storage for ingested originals and pages' files
  attachment/           pages' images and PDFs: upload, sniffing, naming, reading through the page
  export/               the markdown tree out, and back in
  auth/                 passwords, sessions, OIDC verification, service tokens
  api/                  HTTP handlers
  mcpserver/            MCP tool definitions and bearer auth
  testdb/ testidp/      per-package test databases; a signing identity provider for tests
db/
  migrations/           goose SQL migrations
  queries/              SQL for sqlc
web/                    Svelte 5 + SvelteKit static; built to web/build, embedded via embed.FS

internal/domain holds the permission policy as pure functions with no database access, so it can be exhaustively tested against a truth table rather than inferred from integration tests.

Technology choices

Full rationale, with rejected alternatives, in adr/.

Layer	Choice	Why
Language	Go 1.27+	One static binary, no runtime dependencies, goroutines suit the outbox and scheduler
Database	PostgreSQL 17	Recursive CTEs for link traversal, GIN for tags, JSONB for frontmatter, SKIP LOCKED for the scheduler
Queries	sqlc	Hand-written SQL, generated types. The hard queries here are recursive CTEs — an ORM would fight them
Migrations	goose	Plain SQL files, reviewable in a pull request
Driver	pgx v5	LISTEN/NOTIFY, native array and JSONB handling
Search	Typesense 29	Typo tolerance, faceting, string[] filters for ACL, optional vector search
Markdown	goldmark	CommonMark-compliant with a real AST — link extraction must not be a regex over prose
Frontmatter	goccy/go-yaml	Round-trips ordering and comments
Diff	hexops/gotextdiff	Unified diffs computed on read
Sandbox	QuickJS on wazero (fastschema/qjs)	CGO-free. wazero gives a syscall-free substrate; QuickJS enforces memory and execution-time limits internally
Cron	adhocore/gronx	Expression parsing only — schedule state stays in Postgres so replicas stay correct
IDs	UUIDv7	Time-sortable, so index locality and "recent" queries stay cheap
Web UI	Svelte 5 + Vite, adapter-static	Compiles to a static tree that embed.FS takes directly; no runtime Node
Tests	stdlib testing + testcontainers-go	Real Postgres and Typesense — the interesting behaviour is in SQL and index filters

Deployment

One deployment serves one company. Users, documents, and groups carry no org scoping — which keeps every query and every permission check meaningfully simpler. A hosted offering would run one deployment per customer.

podman run -e DATABASE_URL=… -e TYPESENSE_URL=… ghcr.io/torniker/codna serve

Retrofitting multi-tenancy later means touching essentially every query. That is understood and accepted; see adr/0002.

Deployment strategies

Three, all running the same binary — full guides in ../deploy/README.md.

Strategy	Artifacts
Compose	compose.yaml, CoDNA behind an app profile
Kubernetes	deploy/kubernetes/ manifests + kustomization, or deploy/helm/codna/
VM / bare metal	deploy/systemd/ — sandboxed unit plus env template

Builds target OCI rather than a vendor: Containerfile and compose.yaml work with podman, docker, buildah, and kaniko unchanged. See adr/0011.

Two operational facts worth knowing before you deploy:

• Migrations are embedded in the binary. codna migrate up applies the schema from a copy compiled in at build time, so the schema can never disagree with the code running against it, and no deployment ships migration files. Run it as a Job or an ExecStartPre — not as an initContainer on a multi-replica Deployment, since goose takes no lock by default.
• The local storage driver pins you to one replica. Ingested originals need a writable volume and ReadWriteOnce cannot be shared. CoDNA is otherwise replica-safe — the outbox worker and scheduler both claim work with FOR UPDATE SKIP LOCKED — so scaling out needs only ReadWriteMany or an object-store driver.

Milestones

Milestone	Scope
M1	Data model and technology spec (this document set) ✅
M2	Schema, domain, Typesense indexing, outbox worker, sharing ✅
M3	REST API and auth — sessions for the UI, OIDC tokens for agents ✅
M4	MCP server — search, read, write, share-with-suggestions ✅
M5	Web UI — read, edit, history and diffs, sharing, search ✅
M6	Raw ingestion — uploads, paste, bulk load, six converters ✅
M7	Automations — sandbox, triggers, scheduler, run records ✅
M8	Groups and accounts, export and import, CI and release ✅
M9	Manageable by a machine — service tokens, instance identity, status, dead letters, export over HTTP ✅
M10	Web UI, second pass — folder tree and quick open, CodeMirror editor with drafts and conflict resolution, history restore, move and restore, share by tag, related links, automations editor, accounts, settings, admin; one token set for both themes, a phone layout, keyboard shortcuts ✅
M11	Web UI, third pass — one New and one search, an account menu, spaces listed once they hold something; a page for connecting an AI agent (mcp on /api/instance); a home with Get started and Continue reading; Edit, Share and More on a document, its title once; a share dialog that names people and groups, and upgrades a reader; a link picker and writing from source material; an inbox that keeps or restores a link in place and counts the news (kept_at, /api/me/inbox-counts) ✅
M12	Web UI, fourth pass — a folder opens as a page of its own, its folders and then its pages (exact=1 on GET /api/documents), and the tree keeps its place; writing that looks as it reads: markdown shown only around the cursor, a click into the text edits there and Done goes back to the same place, a toolbar over a selection and the usual keys, a slash menu for blocks, a checkbox ticked while reading saved as a version of its own. The editor changes only the characters it must (ADR 0006) and loads only for someone who can write ✅
M13	Images and files in pages (ADR 0015) — pasted, dropped or chosen from /image, kept in <slug>.files/ beside the page and linked as plain markdown; drawn while writing and reading; served through the page's own permission with a sandboxing policy; exported beside the page and imported back; get_file hands an agent the image ✅
Later	Server-side AI workers, outbound webhooks, OCR, connectors, multi-org, git mirror

Auth is deliberately absent until M3. M2 exercises permissions through an Actor value constructed directly in tests, which is enough to prove the rules without committing to an auth mechanism.