Agents and deployment terms
Terms that show up once an AI feature ships: keeping a human in the loop, forcing machine-readable output, and what inference costs when real users arrive.
human-in-the-loop (hitl).
Human-in-the-loop is a system design where a person reviews, approves, or corrects an AI system's outputs or actions at defined checkpoints instead of letting it act fully autonomously. It trades throughput for control at the steps where mistakes are costly.
The pattern predates language models, but agents made it urgent: once software can take multi-step actions on its own, the design question becomes where a person signs off. Google Cloud's overview of the pattern frames it the standard way, humans validating, correcting, and providing feedback inside an otherwise automated loop (cloud.google.com).
Vibe coding already runs on this pattern even when nobody names it. An agent proposing a diff you review before it lands is human-in-the-loop; so is a coding tool pausing for permission before running a shell command, and a content pipeline where a person approves the publish. The craft is placing the checkpoints where errors are expensive and irreversible, and nowhere else, because every checkpoint you add converts machine throughput back into human queue time.
When it matters: production writes, payments, deletes, anything customer-visible, anything you cannot roll back. When it does not: steps that are cheap to verify after the fact, where review-on-failure beats approve-on-every-step. The honest test for each checkpoint is whether the human is genuinely deciding or has become a rubber stamp; a stamp is latency with no safety, and it is better removed in favour of a verifiable check.
structured output.
Structured output is a model response constrained to a machine-readable format, typically JSON matching a schema, instead of free prose. APIs enforce it through JSON modes, function calling, or constrained decoding, letting code consume model output without brittle parsing.
Free-text model output is for humans; the moment code has to consume it, you want a contract. Providers ship this as a first-class feature: OpenAI's structured outputs constrain generation to a supplied JSON Schema (platform.openai.com), and tool-use APIs across vendors do the equivalent for function arguments.
This is one of the quiet reliability upgrades in the vibe-coding stack. Every agent that calls a tool is depending on structured output under the hood: the model does not run your function, it emits a JSON blob naming the tool and its arguments, and the harness executes it. When you build your own features, the same move applies: define the schema first, validate what comes back, and your parser never meets an apology sentence wrapped around the JSON.
The caveat worth knowing: a schema guarantees shape, not truth. A response can be perfectly valid JSON and still carry a hallucinated value in every field, so validation replaces parsing pain, not verification.
ai inference and inference cost.
Inference is running a trained model to produce output, as opposed to training it. Inference cost is what you pay per request, billed per token by API providers or as GPU time when self-hosting, and it dominates the running cost of shipped AI features.
Training happens once at the lab; inference happens every time anyone uses the result, and it is the meter that runs while your feature is live. API providers price it per million tokens, with rates published on their pricing pages, for example Anthropic's (anthropic.com/pricing), and input tokens, output tokens, and cache reads usually carry different rates. Self-hosting swaps the per-token meter for GPU time, which is cheaper only past serious utilisation.
The builder's surprise is that inference cost is a product-design variable, not a fixed tax. Context length drives input tokens, so an agent that rereads the whole repository on every step burns multiples of one that reads only the files it needs. Retries, verbose system prompts, and chatty agent loops all compound the same way, which is why two products with identical features can differ several-fold in cost per user.
When to look at this closely: before pricing your own product, and before an agent loop ships to more than a handful of users. Our pricing comparison of the agent metering models shows how differently vendors pass this cost through, and why the plans are hard to compare on the sticker.