Scaffolds: Versioning the Whole Agent, Not Just the Prompt

When a customer-facing agent regresses, the first instinct is to blame the prompt. So you open Prompt Studio, diff the latest version against the one from last week, and the diff is empty. The prompt didn't change. You shipped a tool change. Or the model upgraded. Or the temperature was bumped during an experiment three days ago and nobody wrote it down.

A prompt is one ingredient. The thing that actually produces agent behavior is the whole recipe — prompts plus tool definitions plus model configuration — and a recipe needs its own version identity.

What a scaffold is

A scaffold is the immutable bundle of (prompts_ref + tool_definitions + model_config). Winnow's backend content-addresses the bundle: the same inputs hash to the same scaffold_id, and any change — a new tool, a temperature bump, a different model — produces a new scaffold with a new hash. A trace records the scaffold it ran under. The scorecard can then attribute a regression back to a specific recipe, not just a specific prompt.

What an alias is

A scaffold hash is a 40-character hex string. Nobody wants to deploy by hex. An alias is a moving label — production, canary, dev — pointing at one scaffold hash at a time. Aliases move when you promote a new recipe; hashes don't. When the regression shows up, the question "what was production last Tuesday?" has a single answer: the scaffold whose production alias was set at that time.

What the page does

/scaffolds is a read-only timeline of every scaffold version published in your workspace, newest first. Each row carries the hash, the alias (colored by convention — green for production, blue for canary, purple for dev), the tool count, and the model name plus temperature. Click a row to inspect the full tool definitions and model config as JSON.

Prompt-level diffs are not on this page. They live in Prompt Studio, where they already lived. The scaffold timeline links the prompts_ref of each row out to Prompt Studio, so you read the timeline to find which scaffold shipped when and click through to Prompt Studio when you need to see what changed inside the prompt itself.

What's deferred

Three things from the design aren't built yet, and the page is honest about it.

• Per-hash pass rate. The mockup shows a pass-rate column on each timeline row, joining traces to scaffolds by scaffold_id. The aggregation isn't shipped; rendering a placeholder would be dishonest, so the column is omitted entirely until the backend lands.
• Move alias here. Today the only way to retarget an alias is to publish a new scaffold with that alias. A dedicated POST endpoint that re-points an alias to an existing hash is the next slice; the button is intentionally absent from the timeline.
• One hash, many aliases. A scaffold currently carries one alias label per storage row. A canary-then-promote rollout (same hash gets canary first, then production added without removing canary) needs a join table; that's the third follow-up.

Where to start

If you've never published a scaffold, the page shows the SDK snippet that wires the publish call. If you have, open /scaffolds and look for the row whose alias matches the one your traces reference. The next time something regresses, the timeline answers "what shipped, when" before you even open a trace.