Reliable AI Workflows: Structured Outputs and Tool Calls

Turn a model's suggestions into checked data and limited actions, with clear handling for the times it fails.

Content date: October 15, 2026.

Plain English

Structured output makes a model's reply follow a shape you define, such as a category from a fixed list plus a ticket number. Tool calling lets the model propose an operation and its inputs, which your app then decides whether to run. Neither one proves the content is true or the operation is allowed. Your server still has to check business rules and permissions, and decide whether the action may run.

Use this when

Use this when model output goes into a database, drives a workflow, or proposes an action, instead of only being shown as text.

If you are just starting

Start with a read-only classifier and a small schema (a description of the allowed shape). Check the output at the edge of your app. Show a refusal, timeout, cut-off answer, and failed check as separate states. Use recorded sample responses for your first app tests, and label them as samples, not live model results.

When your project grows

Add signed-in tool use, an approved list of operations, limits, safe repeats (idempotency), and audit records, and check permission again right before an action runs. OpenAI, Anthropic, and Google each document these features with different parameters and schema support, so read your provider's current page. Check why each response ended: a refusal or a cut-off at the token limit may not match your schema. For new OpenAI projects, use the Responses API; OpenAI shut down the older Assistants API on August 26, 2026.

Worked example

At a property-management company, a support classifier returns {"category": "maintenance", "ticketId": "T-4821"}. The category is valid, but T-4821 belongs to a different tenant company. The server compares the ticket's owner with the signed-in user's company and rejects the update. Valid JSON never overrides that check.

Practice

Validate a proposed ticket action

  1. Define a schema with a small category list and a ticket ID.
  2. Create sample responses: valid, malformed, refused, timed-out, other-owner.
  3. Write server-side validation and ownership checks that run before any update.
  4. Submit the same allowed request twice and show that only one update happens.
  5. If you can afford a live model, test it and label those results separately.

Deliverable: A working validated workflow, a table of failure states, and proof that other-owner and duplicate requests can't make unintended changes.

Common mistakes to watch for

  • A classifier returns a well-formatted but wrong category, and it triggers a refund email.
  • The model supplies another customer's ticket ID and the app updates it.
  • A timeout triggers a retry, and the customer is charged or emailed twice.
  • A refusal or cut-off response is stored as a finished answer.

Prompting

A weak prompt

Return JSON, then execute whatever action it says.

A better prompt

Design this AI workflow for [my support inbox] with a small output schema and a separate server-side action handler. Validate the shape, the business rules, and that the signed-in user owns the record. Handle refusal, timeout, cut-off output, and invalid arguments as distinct states. Allow only named tools, and make repeated requests safe. Test with recorded samples first, and record live-provider behavior separately if I have a free or local option.

Uncertain model output stays separate from decisions your application makes, and failures become visible instead of silent.

Check before you move on

  • The server validates the schema before using model output.
  • Ownership comes from the signed-in user, not the model.
  • Your code enforces the business rules.
  • Refusal, timeout, cut-off, and invalid states are handled.
  • Each action's side effects are limited and logged.
  • You tested that retries don't repeat an action.

You are done when

  • Invalid and unauthorized requests fail and change nothing.
  • The allowed action works, and a duplicate request is harmless.

Questions to think about

The model returns a valid schema with another tenant's ticket ID. What should the server do?

Deny the action. The schema check confirms the format, while the ownership check decides whether this user may touch that ticket.

Does structured output guarantee the model's category is the right one?

No. It only forces the shape of the answer, so you still need to test accuracy on realistic examples.

A request times out after the ticket may already have been updated. Should the app simply try again?

Not blindly. Use an idempotency key or another durable duplicate guard and check what was stored, since the caller can't know the final state.

Official references