Agent docs

Authoring an agentic module

The doc the owner says keeps getting rewritten in every app: how an agentic capability (a model in…

The doc the owner says keeps getting rewritten in every app: how an agentic capability (a model in the loop: enhancement copy, image generation, attribute normalization, part-type matching) is set up, structured, interfaced, and layered, and what the options are at each layer. It sits beside 00-building-a-module.md (which covers any module) and adds what is specific to payloads that call models. It is a recipe book, on purpose: the owner's stance is that "we cannot delude ourselves by saying that we will build the end-of-history FINAL version of each"; so where the four V1 modules diverge, the divergence is written as a recipe with its ingredients showable, not smoothed into a standard.

Vocabulary naming test two: everything here is a payload on the runner, with fittings reached through Context, reference data for taxonomies and prompts that are published, and the divergent bits labelled bespoke or filed as recipes. Two words fit without strain; one friction, noted below: prompts sit between "reference data" and "system config" and the words do not pick for you.

Audience: an agent or a person writing enhancement, image generation, attribute normalization, or part-type matching as a module, and whoever reviews it.

What is the same every time, so it is factored#

The owner's observation is that "SO much of it is still the same and well quantifiable". This is that list, and every line is something the runner or a fitting provides, so the payload never writes it:

  • one item in, one outcome out, retries owned by the runner (canon/03, canon/04)
  • every model call traced (ctx.trace), rate-limited and breakered (ctx.limit, ctx.guarded), metered by tokens and cost (ctx.meter, contracts/usage-event.md), and cached by input when the manifest says (ctx.cached)
  • confidence and evidence on every result, and needs_review with a stable reason instead of a guess (canon/10)
  • reference data loaded as a versioned snapshot, version stamped on the outcome (canon/05)
  • a per-item log a caller reads without a shell (canon/07)
  • a manifest that names params, item, output, codes with labels, variants, meters, and outputs (contracts/manifest.md)

If a payload finds itself writing any of these, stop: it is a runner gap, add the verb (contracts/runner-verbs.md).

The layers of an agentic payload#

Every one of the four modules has these layers; what differs is the recipe in each. Name the layer, then pick or record the recipe.

item ──► prepare ──► retrieve ──► generate ──► self-check ──► judge ──► settle ──► outcome
│ │ │ │ │ │
normalize reference model call rules, second result |
input, data, web, with prompt schema, opinion needs_review |
templates cache + params confidence (variant) error
LayerWhat it doesOptions seen in the estate (recipes)Where it lives
prepareturn the item and params into what the model needs: template substitution (<<Column>>), field selection, normalization of the part numberApp V5 template DSL with <<llm:topic=…>> spans; speedway's frozen content template per job; walmart's spec-sheet mappingpayload; templates are per-tenant config passed in params
retrievewhat the model is allowed to know: reference data (taxonomy, PCdb, accepted values), web research, prior outcomesApp V5 research mode via Vertex grounding; speedway scrape then normalize as two runs; walmart scrape stage feeding contentpayload behind ctx.reference, ctx.cached; research as a variant (self or extractor)
generatethe model callGemini via Vertex (speedway, walmart); OpenAI and Gemini for images (App V5); model chosen per variantpayload, always through ctx.trace and ctx.guarded; model id in the manifest's variant options
self-checkstatic rules and a self-reported confidence: length limits, banned characters, must start with the part type, schema validity, "unverifiable claim"App V5's RESEARCH_QUALITY_LOW; the owner's draft's Goal classes (satisfactory, needs review, retry, failed)payload; static checks are code, confidence is judged_by: self
judgea second opinion when self-report is not enougha chained judge capability inside the module, a standalone judge module, or the app's owna variant; ctx.judge; the manifest says which score is authoritative (adr/004)
settledecide the outcome: result with confidence, needs_review with reason and evidence, or terminal errornet-new attribute values held for review (attribute normalization); ambiguous part type held with candidates (matching); low-confidence copy held with the draftpayload returns the outcome; the runner records attempts, usage, timings

Options and recipes, by module#

The four V1 modules, each with what it shares and what is its own. Each "own" line is a recipe: name it, keep it, do not pretend the others do it.

ModuleSharedOwn recipe (ingredients to make showable)
enhancement (content)prepare, retrieve, generate, self-check, settletemplate DSL and per-template rules (App V5); research mode as a variant; multi-version copy as three jobs or one job with a versions param; outputs transform to a load-sheet shape
image generationgenerate, self-check, settleprompt built from context fields; reference-image search (Oxylabs) as a retrieve recipe; render with OpenAI gpt-image-* or Gemini; validation votes as a judge variant; artifacts to the module's own bucket, not S3 in another region (canon/05 lapse)
attribute normalizationretrieve (accepted values snapshot), generate, self-check, judge, settleper-tenant accepted-values snapshot as reference data; net-new value detection as a needs_review reason; aggregator over many sources as a run_batch recipe
part-type matchingretrieve (taxonomy snapshot), generate, judge, settletenant taxonomy vs PCdb as a variant; candidates list on needs_review; a judge.parttype variant when self-report is weak (owner: self-report "tends not to be very good in some cases")

Prompts and templates: the one place the words do not pick#

Prompts are versioned text a model reads. They behave like reference data (published, versioned, the version recorded on the outcome) and like system config (they change with the payload's code and its evals). The recipe that has worked (App V5's PROMPT_MGMT_ENV, its prompt management behind a key) is: prompts are versioned artifacts the payload loads by name and version, the version is stamped on every outcome, and a prompt change is a payload minor bump; per-tenant template text (a customer's copy template) is tenant config passed in params, never baked into a prompt. Write which one a given text is in the module's README, because the vocabulary will not.

Evals are part of the module#

An agentic payload without an eval set is a guess with a deploy. The minimum is a fixed set of items with expected outcomes (or acceptable ranges) run in env: local by the conformance CLI as an extra group, reporting pass rate, confidence calibration (self-report versus expected), cost per item, and duration. A change to a prompt, a model, or a variant default runs it before merge. Where the eval items come from: resolved review items are the teaching loop the business note wants, and the app is where they accumulate; the module ships with a seed set and grows it from resolutions the app hands back.

What is bespoke, and how it is kept#

A tenant-specific rule inside a payload (JEGS's title format, Zeder's banned words) is bespoke: it goes in a labelled place (bespoke/<tenant>/ in the module or, better, in the tenant's params passed by the app), it is named in the instance doc, and it has a review date at which it becomes a variant, a param, or a rule everyone gets. The App V5 lesson is that the hardcoded version worked and cost architectural headroom later; the recipe is to keep it and label it, not to forbid it.

Order of work for one agentic module#

  1. Manifest first: params, item, output schemas; codes with labels; variants (model, research source, judge); meters; outputs.
  2. Reference data as a snapshot with a version, loaded through ctx.reference; prompts as versioned artifacts.
  3. The payload's run through the layers above, every outside call behind a Context verb.
  4. A seed eval set and its conformance group, green in env: local.
  5. Instance doc with the module's recipes named under "Own", and its Lapses.

Do-nots#

  • Do not import a search client or a tracing client in a payload; reach them through Context. A model client may be imported, and every call through it is wrapped in ctx.trace and, for a vendor with a limiter, ctx.guarded (../contracts/runner-verbs.md), which is what makes the call visible and budgeted.
  • Do not return a confidence you did not compute or receive.
  • Do not guess when unsure; needs_review with a reason and evidence.
  • Do not bake a tenant's template into a prompt; pass it in params.
  • Do not ship a prompt or model change without running the eval set.
  • Do not smooth a module's own recipe into another module's; name it, keep it, make its ingredients showable.
@versable-git/ui · reference, canon, and method, read in place