Describe, don't route

The release

The previous entry ended with two facts in tension. The signature could not drift from the interface, because it was the interface; and precisely because of that, the interface could not be read. There was nothing to hand a caller, a router or a document generator — only the value's shape and the rules for walking it, applied again on every request. The first intuition — at Lyft, before there was a project to carry it — pointed at the opposite arrangement: what a service offers, existing as an artifact in its own right.

This entry walks the line's first step, version 1.0.0¹, where the interface became a value. The release's Readme states it in one sentence:

The API is expressed once as Go functions. Runtime routing, the committed OpenAPI contract, interactive documentation, and the generated CLI are projections of that same definition.

Two moves, in that order: first the description exists, then everything else consumes it — one common representation that builds the service, statically or by reflection, and generates the documentation and the CLIs:

flowchart LR
    api["Go API functions<br/>+ neutral metadata"]
    plan["the common representation:<br/>one description of the boundary"]
    serveStatic["serve statically<br/>(generated direct calls)"]
    serveReflect["serve by reflection<br/>(the fallback)"]
    docs["generate the documentation<br/>(OpenAPI document, Swagger UI)"]
    clis["generate CLIs"]

    api -->|"CompileAPI"| plan
    plan --> serveStatic
    plan --> serveReflect
    plan --> docs
    plan --> clis

The description

An API value may now carry two more values beside its functions. One says where the operations live:

func (*Polls) Routes() map[string]string {
    return map[string]string{
        "ListPolls":   "/polls",
        "DeclarePoll": "/polls",
        "Cast":        "/polls/:poll_id/ballots",
        "ReadLog":     "/polls/:poll_id/log",
        "ClosePoll":   "/polls/:poll_id",
    }
}

The other says how each one is reached and what it promises: APIDocumentation, keyed by method name, with an entry per operation that can set the HTTP method, the success status, the operation's name, a summary, parameters, non-inferable responses and security. The polls service uses it to fix the cost the previous entry recorded — a read forced into POST because the signature had a request:

func (*Polls) APIDocumentation() fb.APIDocumentation {
    return fb.APIDocumentation{
        Name:        "polls",
        Description: "Preference polls with cryptographic guarantees.",
        Operations: map[string]fb.OperationDocumentation{
            "DeclarePoll": {SuccessStatus: http.StatusCreated},
            "ReadLog": {
                Method: http.MethodGet,
                Parameters: []fb.ParameterDocumentation{
                    fb.ParameterOf[string]("poll_id", fb.ParameterInPath, "The poll's identity.", true),
                    fb.ParameterOf[int]("page", fb.ParameterInQuery, "The page to read.", false),
                },
            },
        },
    }
}

ReadLog is a GET at /polls/:poll_id/log, and this time something reads the fact: the route is registered from the description, and the same fact reaches the OpenAPI document without being written twice. The signature still defaults the verb, for compatibility; the description can now say what the signature cannot.

Both carriers are metadata-only, and the reason is mechanical: a generator never constructs the service. At build time it holds the type, not an instance — no configuration to give it, no database to connect — so the machinery is handed a typed nil, (*Polls)(nil), and the metadata methods run on it. Go permits exactly that: a method may be called on a nil pointer as long as it never dereferences its receiver, and a Routes or APIDocumentation that returns literals never does. The interfaces' own comments say so — "metadata-only so the method can safely be called on a typed nil receiver by build-time generators" — which turns a Go curiosity into the discipline: metadata never touches the receiver, and the functions stay untouched. What changed is that the interface left the code's shape and became a thing: constructible, checkable, readable by tools that never see the functions.

It is also refereed. Describing an API validates the description: unsupported verbs, duplicate operation IDs, parameters without names, locations or Go types, a success status outside 2xx, a 204 on an operation that returns a value, security that contradicts itself, documentation naming a method the API does not have — all refused, at description time. The previous entry pointed at registration refusing shapes nobody could call as the library's first no; this release grows the same gesture to metadata that lies.

What the description does not change is the leak. ReadLog drops its request argument — a read either takes a body it should not have or reads the context — and does the second:

func (*Polls) ReadLog(ctx context.Context) LogPage {
    pollID := http_common.PathParametersFromContext(ctx)["poll_id"]
    page := 0
    if raw := http_common.QueryFromContext(ctx).Get("page"); raw != "" {
        parsed, err := strconv.Atoi(raw)
        if err != nil || parsed < 0 || parsed > 500 {
            panic("page must be between 0 and 500")
        }
        page = parsed
    }
    // ...
}

Read the example for what it shows: the function holds path parameters and query values, untyped strings straight from the wire, parses them by hand and refuses with a panic. The context made the signature clean — nothing of HTTP appears in it — and that is precisely how the transport leaks now: not through the signature anymore, through the context, hidden in it. The description can even state Required, Pattern and Enum on a parameter — and publishes them to the document — but nothing enforces them: true for readers of the document, silent for callers of the API. Keep both pictures; the next entry exists to close the leak, with the request built from a contract instead of handed through a context.

The abstractions underneath

Serving a described API is consumption, not derivation. In the code, the common representation of the diagram is two declarations:

// CompileAPI resolves an API to the common execution plan. Generated plans
// are preferred when present; reflection remains the complete fallback.
func CompileAPI(api any) (compiled CompiledAPI, err error)

// CompiledAPI is the normalized execution plan consumed by transports. The
// Description is also the single input used by OpenAPI renderers. Invokers are
// keyed by the exported Go method name in Description.Operations.
type CompiledAPI struct {
    Description APIDescription
    Invokers    map[string]OperationInvoker
}

That doc comment is the simple diagram said in code: one description, and every consumer — the transport that mounts the service, the definition that packages the document — reads the same value. The process, with the components that make it possible:

flowchart TD
    api["Go API functions + neutral metadata"]
    compile["CompileAPI"]
    plan["the common representation:<br/>one description, one invoker per operation"]
    adapter["transport adapter<br/>mounts the service (HTTP today)"]
    definition["the app's build-time definition"]
    artifacts["static Go + OpenAPI + lock manifest"]
    swagger["Swagger UI"]
    clis["CLIs (api-cli)"]

    api --> compile
    compile -->|"generated invokers, behind a build tag"| plan
    compile -->|"reflected invokers, the fallback"| plan
    plan --> adapter
    plan --> definition
    definition --> artifacts
    artifacts --> swagger
    artifacts --> clis

An invoker is a transport-neutral function: it receives the context (which carries the headers, query values and named path parameters) and a way to decode a body, independent of any router. AddApi compiles, then registers exactly what the plan says, at operation.Route under operation.HTTPMethod. The walk over the value that used to be the serving is now a build step that produces a value; the router no longer knows how to discover an API, only how to consume a plan.

The plan has two interchangeable compilers, and neither is trusted for being either one. A generator emits reflection-free invokers — "reflection is used only while running the generator; the emitted provider calls API methods directly" — behind a build tag, and CompileAPI prefers the generated plan when present. Then it validates that plan with the same rules as a reflected one: the generated description passes through the same referee, and the plan must have an invoker for every operation and none for anything else. A fixture in the repository — a real generated consumer, built to prove the reflected and the generated execution paths stay equivalent — serves the same requests through both compilers and fails if the answers drift.

The document gets the same treatment. The build-time definition of the app — "the single build-time description of one functional app" — derives the static Go, the OpenAPI document, and a manifest that "records exactly which committed artifacts were derived from a service definition", each with its SHA-256. The artifacts are committed, and a test regenerates them and fails if what is committed is not what the definition produces now. That is where the first entry's promise becomes ordinary: the OpenAPI document cannot drift from the code that serves it, because both are projections of one description — and a machine checks it.

Two compilers, one plan

Go has two standard answers to "how does a framework call code it did not write", and teams usually pick one the way they pick an editor. Reflection walks the value at runtime: no build step, nothing in the repository but the code itself, and the plan derived again at every startup. Generation runs at build time and emits ordinary Go: a build step, a file in the repository, and invokers that call the methods directly.

Look at what each compiler actually produces for the same operation. The reflected side never names the method it is calling. The request arrives as a reflect.Value the handler allocates and binds into; the call itself goes through the machinery — methodValue.Call(in) — types erased, the method discovered at boot:

newValue := reflect.New(t)
if err := c.BindJSON(newValue.Interface()); err != nil {
    return nil, err
}
...
values, iErr := UnBindError(func() []reflect.Value {
    return methodValue.Call(in)
})

The generated side is a function any reviewer can read, committed beside the code it serves. This is the fixture's emitted invoker for the same operation, verbatim — the method is named, the argument is typed, and the nil-pointer case is spelled out where the reflection machinery used to handle it invisibly:

"Create": func(invocation fb.Invocation) (fb.InvocationResult, error) {
    var request domain.Request
    if err := invocation.Decode(&request); err != nil {
        return fb.InvocationResult{}, err
    }
    value := api.Create(invocation.Context, request)
    if value == nil {
        return fb.InvocationResult{Value: nil, HasValue: true}, nil
    }
    return fb.InvocationResult{Value: *value, HasValue: true}, nil
},

The costs are paid in different currencies. Reflection re-derives the plan at every startup of every instance — same value in, same plan out, the work repeated and shown to no one. The plan exists only inside the running process: nobody can diff it, review it or digest it, and a mistake in the derivation ships inside the binary with no witness but serving behavior. Generation turns the same derivation into a file — diffable in review, hashable into the lock manifest, regenerable and compared byte for byte by a test — and in exchange keeps a second thing that must be kept true of the first. Each failure mode becomes a check instead of a hope: staleness is TestStaticFixtureArtifactsAreCurrent, and disagreement between the compilers is the parity fixture serving the same requests through both.

Why keep both, if the file is the one with receipts? Because they serve different operations of the same truth. Reflection is "the complete fallback": a service with zero generated artifacts still compiles and serves, which a repository that owes nobody a build step may prefer. Generation commits its plan and lets the machine hold it honest. CompileAPI takes them in that order — "generated plans are preferred when present; reflection remains the complete fallback" — and trusts neither for being either one: both descriptions pass the same referee, both plans answer to the parity fixture.

With one description as the source, reflection and generation stop being rival architectures and become two readers of one fact — the OpenAPI document already a third — and choosing between them is operational: boot the derivation, or commit it. The fact does not move when the reader does.

The same document as annotations

The idiomatic way to get an OpenAPI document out of a Go service is annotations:

// @Router /polls/{poll_id}/log [get]
// @Param page query int false "The page to read."
func (*Polls) ReadLog(ctx context.Context) LogPage { ... }

A tool walks the comments and emits a document. The comments are not code: the compiler never reads them, the router never reads them, and nothing connects them to the route actually registered except the discipline of the person writing both. The description here is a value the compiler checks, the router consumes and the document renders — one fact, three readers that cannot disagree because they read one thing. The schema-first alternative was met two entries ago: there the interface is real but the domain speaks the schema language's types. Here the Go stays the source and the interface becomes a value beside it.

What the tests record

• TestDescribeAPIRejectsDuplicateAndInvalidMetadata
• TestDescribeAPIValidatesSuccessStatusSemantics
• TestCompileAPIPrefersGeneratedPlan
• TestCompileAPIValidatesGeneratedPlan
• TestStaticFixtureExecutionPlan
• TestStaticFixtureHTTPParity
• TestStaticFixtureArtifactsAreCurrent

The last two are the release's signature: the reflected and generated services must answer identically, and the committed artifacts must be what the definition produces now.

What this release did not solve

The document can state what the runtime does not check: Required, Pattern, Enum are published and enforced by nobody, and the function reads strings from the context and parses them by hand, refusing with a panic and a 500. The request is not built from the description — the body arrives as a decode callback, everything else as untyped context. There is no security: nothing authenticates a caller, nothing authorizes one, no principal exists anywhere in this release. Security on an operation's documentation shapes the OpenAPI document — and the referee does check that field for self-contradiction — but that field is the only place the word appears: the plan carries no requirement, and nothing is checked before the function runs. Who may call is documentation. Failure is not a fact; Responses documents error responses as prose. And the description itself speaks HTTP — verb, route and status live inside it — so "neutral" here means neutral to the HTTP server, not to the wire. Each of those is a later step of the line.

Where the real service stands

On today's Koba this release's promises are boring furniture: the plan is compiled, the OpenAPI document generated from it, both committed beside a lock manifest, CI refusing a stale artifact. What moved afterwards is where the facts live. Route, verb and status sat in the description at this release; today they belong to the wiring, decided at the composition root, and the contract's own header says it "names no route and no status". The description became two values — the contract, which says what is true of the boundary, and the wiring, which says how it is reached. The build-time definition survives the whole line, renamed and slimmed into a driver: it writes, hashes and checks the committed artifacts, and what an artifact says is the wire's business — its own comment today is that it "names none" of them, and it holds no OpenAPI dependency, no wire's vocabulary at all. The OpenAPI renderer did what the fifth step demands of everything wire-shaped and left the core: it lives in the HTTP wire now, rendering the document it commits. What this release could only document, the contract era began to enforce: the page bound of zero to five hundred is checked before the function runs, not promised in a schema and parsed by hand inside. The increment still does not verify the proof, draw the sample or aggregate — each is a later increment, and each will have its entry.

A system compiler

Seen from this release, the line's shape comes into focus: Koba is not a web framework that happens to generate files. It is a compiler whose source is the described API and whose target is a running system. The signature entry moved the interface into the code; this release moved the derivation out of the runtime. Type-checking is the referee that refuses a description that lies; code generation is the committed plan; projection is the OpenAPI document; freshness is the digests and the checks that hold every artifact honest. Each is a pass over one source, and each produces an artifact a machine verifies. Serving, which began as something the framework did at runtime, became what the compiler emits.

The two compilers of this release were the same story at small scale: reflection and generation were never rivals about truth, only about when the derivation runs. The rest of the line generalizes that answer — every entry that follows adds a pass or widens the target: contracts type-check the boundary, wires multiply the targets beyond HTTP, and by the end the core names no wire at all. A compiler, in the end, is a machine for making derivations explicit. Koba's claim, entry by entry, is that a system deserves one too.

Where this blog goes

The next entry walks the release where the boundary truth became a value of its own: a contract beside the functions, typed bindings that build the request before the function sees it, and the end of reading strings from the context.