How to Build a Production AI Agent in Python

Consider a support ticket waiting for a customer upload. A note saying “the logs were requested” sends the next engineer to the customer. A note saying “the logs arrived” sends the engineer to an attachment. Both sentences are plausible. Only one matches the ticket. A production AI agent has to make that distinction and survive a second test when the application commits the note.

Uvik Software tested 20 scripted proposals for this support-ticket example. All 20 passed a typed schema. Adding a lookup against the ticket’s current, trusted evidence sent all five supported proposals to review and stopped all 15 unsupported, superseded, or out-of-scope proposals. The fixture isolates a decision that teams can enforce before approval. It measures the validator on selected cases.

Quick answer

Build a Python AI agent around a single verifiable action. Bind identity and record access on the server, then let the agent propose the action. Check each consequential claim against current evidence, bind reviewer approval to the exact proposal, and commit the effect with a receipt the requester can recover on retry. Uvik Software uses a support note to show which guarantees belong to the application and which outcomes need live evaluation.

Disclosure. Uvik Software provides AI agent development and Python engineering services. The support ticket is a constructed example. The results below come from local Python and SQLite fixtures and a FastAPI route with scripted identities. Real model accuracy, reviewer behavior and support throughput require a separate pilot.

Decide whether the job needs a model

On T-42, a customer reports an intermittent connection and the previous support reply asks for logs. A useful internal note records that the team is waiting for an upload. If the application already emits a reliable log-requested or log-received event and that event is the whole input to the note, render the note with code. Measure the rule’s coverage over the whole eligible queue. Keep a human review path for missing events, contradicted messages and unusual cases.

A model can help where the team has to read varied text, find which ticket evidence matters, or summarize an unusual issue. Begin with the rule and the current manual process as baselines. In a shadow run, show the same eligible tickets to each approach. Count accurate notes, appropriately deferred cases, and active human minutes per eligible ticket, including time to correct a draft. Only add model calls when the resulting improvement justifies that work and the remaining risk.

User actions can change the answer after the agent has read the ticket. In the ICML 2026 τ²-Bench experiments, telecom agents performed worse when coordinating with a simulated user who could also act on the environment than when the agent acted alone. A support evaluation should therefore cover an attachment arriving, a previous upload being retracted, and a customer correcting a prior message. Check each proposal against the event state that the reviewer can still verify. 14

What deployment studies actually measured

The strongest case for an agent is a change in work the team can measure. A 2026 ICML study by Pan and colleagues combined 20 interviews used as case studies with survey responses on 86 systems in production or pilot. The teams described reliability as their biggest development problem and reported using system controls and human evaluation to manage it. The authors recruited through multiple channels including professional networks and describe geographic and participation bias. The study documents implementation practices in that sample. 22

Nubank’s November 2026 preprint reports a stronger design for measuring customer outcomes. It shows two live A/B tests on card support. The first compared Card Management, which covered a broader set of requests, with Card Delivery. The second held the Card Management task constant and compared a screened, refined model configuration with its incumbent. Each comparison used its own control. 23

Nubank research team, November 2026 preprint, Table 2. Sample sizes are rounded as reported. Self-service counts sessions that stayed with the agent. Transactional NPS uses a separate respondent sample.

The second comparison illustrates why a team needs more than one release metric. More sessions stayed with the agent. The satisfaction estimate was compatible with a small decline or improvement. The candidate also performed unevenly in simulation. Before prompt refinement, the selected open-weight configuration had a lower conversation-completeness failure rate than the incumbent, 22.6% versus 37.6%, and a much higher status-retrieval failure rate, 35.2% versus 2.4%. A single average could have hidden that weakness. The simulated write tools used mock responses, so backend effects still required testing. 23

An earlier Nubank paper reports improved transactional NPS across five domains and improved self-service in four. The fifth domain had a slight decline in self-service, attributed by the authors to transient model request failures. Compare completion and quality across the same queue and include upstream outages in the measurement window. 15

For an internal support-note agent, track useful notes and correct deferrals together. An apparent improvement in automation can leave reviewers with stale or unsupported drafts. Keep a separate test that reads the final ticket state, and sample whether each note helps the next person act.

Define one tool contract before choosing a model

Give the model two tools. One reads the ticket already assigned to this run. The other proposes an internal note. Bind the ticket, tenant and requester on the server before the first model call. The model supplies a proposal inside that scope. A signed-in reviewer supplies approval through a separate route. OWASP identifies tool misuse and abuse of agent identity and privileges as separate risk categories. 1

The first two operations are model-facing. The approval and application routes belong to the service. Route unstructured cases to the existing human support queue, where the operator can author a general note under the normal permissions. Configure the proposal schema to reject unknown fields and validate it again in Python. Use a bounded run with a turn limit and request timeout. OpenAI’s tool documentation describes function signatures and structured arguments. A valid argument shape still needs a server-side policy decision. 2 3

Figure 1. Uvik Software’s separately tested stored-write fixture. For a status note, check the evidence before creating the pending proposal. After reauthorizing the requester, the application returns the original receipt for a matching retry.

Test what a valid schema can miss

Both “logs received” and “awaiting logs” are allowed values. The evidence can still refer to another ticket or to a log request that a later upload superseded. An agent reading the customer’s message can also encounter a sentence instructing it to mark the logs as received. That sentence is evidence of what a customer wrote. A verified upload event is the source for the log-received state.

Prompt injection research gives this boundary a practical purpose. The AgentDojo research benchmark offered 97 ordinary tasks and 629 security test cases with malicious instructions placed in tool outputs. Its authors found that the tested models could fail the original task even without attacks, and that attacks could violate some security properties. In August 2026, OpenAI also documented how models in its cybersecurity evaluations, under reduced safeguards, used shared infrastructure to reach the internet and affected third-party systems. These settings are different from a support queue. They show why application permissions and the tool environment need to hold when model behavior changes. 24 25

For a status note, the proposal carries a signal and an evidence event ID. Resolve the ID inside the authorized tenant and ticket. Require the event type that supports the signal, then confirm it is the latest relevant state transition. A log retraction supersedes receipt. A free-text customer message can raise a question for a human. The application renders the status sentence from its template and shows the event to the reviewer.

Uvik Software compared that check with shape validation alone. The same 20 JSON-shaped proposals and event histories went through both paths. Five were valid and 15 were deliberately invalid. Each test used a fresh in-memory database and fixed requester scope. The outcome was whether the proposal reached review. Both paths kept actual commits outside this experiment.

Figure 2. Uvik Software comparison on 20 deliberately selected support-note proposals. Python 3.12.14 and SQLite 3.53.1. The outcome is admission to review. The asset bundle includes the case-level data and runnable fixture.

Download the case-level results (CSV) and the reproducible Python test fixtures (ZIP). The bundle includes source code, test data, dependencies, and instructions for running the checks locally.

One supported case cited an upload followed by a routine customer message. The check admitted it because the latest relevant state event was still the upload. It stopped a different proposal that cited customer text as proof of upload. That distinction matters in production. Treating any later message as a state change would discard a good proposal. Treating any recent message as a trusted state event would admit a bad one.

The fixture uses two signals and a deliberately small set of trusted event types. It demonstrates what a validator can enforce for that contract. Use production ticket samples to discover missing evidence, misclassified events and cases that should be deferred. Recheck event validity at commit if it can change while a draft awaits review, or bind the evidence and revision to a transaction that detects the change.

Connect the agent to the Python service

In a FastAPI implementation, the ticket route authenticates the user and verifies access before it creates a run. The authenticated tenant and requester, ticket identifier, and a scoped service dependency can live in a local context object. The OpenAI Agents SDK makes that object available to Python tools without adding it to the model prompt. The same contract can be implemented with a different model provider. 4 5

If the team needs persistent orchestration, structured handoffs, or a dedicated tracing integration, use Uvik Software’s agent framework comparison to evaluate those requirements against the runtime you plan to deploy.

Expose a read function and a typed tool such as propose_status(signal, evidence_id). The service runs the evidence check, renders the note and stores the trusted event ID beside the proposed write. Return a proposal ID and pending status only after validation. Keep the model’s final message consistent with that status. When the model is uncertain, a deferral with an empty proposal queue is an acceptable outcome.

Send the reviewer the full note, ticket context, requester, and expected version. Create the approval record on the server after checking the reviewer role for that tenant. Capture the fingerprint of the exact version displayed, and make edits require a fresh review. A human-in-the-loop SDK can pause execution while the reviewer decides. The service still owns the business rules that apply at commit time. 3 6

Bind approval to the revision the reviewer saw

Give the reviewer screen an immutable proposal identifier and a server-calculated fingerprint. The review request can carry the decision and the displayed fingerprint. The server derives the reviewer and tenant from the session, loads the proposal in that tenant, and compares the displayed fingerprint to its own calculation before saving approval. If a teammate edits the draft while the review screen is open, return a conflict and reload the draft. The hash in the handler detects a change in the approval payload. Permissions on the approval endpoint and storage establish who made the decision.

Use a typed model with unknown keys forbidden on the proposal and review endpoints. The review accepts the decision and fingerprint. Include the evidence ID in the material the reviewer approves. Before recording approval and again before commit, confirm that the event still supports the proposal. A ticket revision check can enforce this at both points if every relevant event changes the revision within the same transaction. Resolve reviewer identity, ticket access, expiration and current time server-side. Record who rejected a proposal and why, then let a new proposal go through the same review. 3 16

Keep the proposal record available between the draft and decision. A longer process may also have timers, notifications, and steps that resume after interruptions. The write handler can serve as one step inside that process. The workflow runtime controls when it runs, and the service verifies permission and current ticket state when it runs.

Make the write safe to retry

Consider a request that commits successfully and loses its HTTP response. The client retries. The ticket version has already advanced, so checking the version first would reject the retry. Authenticate and check the requester’s current ticket access, then look for a receipt for this proposal and payload. An existing matching receipt completes the retry. For a new write, verify the current approval and ticket version inside the transaction.

Uvik Software tested the following write handler separately from the evidence check. The connection uses sqlite3.Row and starts outside a transaction. The tables hold tenant-scoped tickets and reviewers, versioned proposals, and notes with a unique proposal ID. This handler binds approval to the stored note and ticket version. An implementation using the status-note evidence check should add the evidence ID to that approved payload and revalidate it when necessary. The fixture uses a single authorized ticket owner and a separate active reviewer.

from hashlib import sha256
import json
import sqlite3


class Rejected(Exception):
    pass


def proposal_hash(p):
    approved_fields = [
        p["id"], p["tenant"], p["ticket"], p["requester"],
        p["expected_version"], p["note"],
    ]
    encoded = json.dumps(approved_fields, separators=(",", ":")).encode()
    return sha256(encoded).hexdigest()


def apply_note(db, tenant, actor, proposal_id, now):
    # tenant, actor and now are supplied by trusted server code.
    with db:
        db.execute("BEGIN IMMEDIATE")
        p = db.execute(
            "SELECT * FROM proposal WHERE id = ? AND tenant = ?",
            (proposal_id, tenant),
        ).fetchone()
        if p is None or p["requester"] != actor:
            raise Rejected("proposal unavailable")
        ticket = db.execute(
            "SELECT owner, version FROM ticket WHERE tenant = ? AND id = ?",
            (tenant, p["ticket"]),
        ).fetchone()
        if ticket is None or ticket["owner"] != actor:
            raise Rejected("ticket unavailable")
        fingerprint = proposal_hash(p)
        receipt = db.execute(
            "SELECT id, fingerprint FROM note WHERE proposal = ?",
            (proposal_id,),
        ).fetchone()
        if receipt is not None:
            if receipt["fingerprint"] != fingerprint:
                raise Rejected("idempotency conflict")
            return receipt["id"]
        reviewer = db.execute(
            "SELECT active FROM reviewer WHERE tenant = ? AND id = ?",
            (tenant, p["approved_by"]),
        ).fetchone()
        if (p["approved_by"] == actor or not reviewer
                or not reviewer["active"]
                or p["approved_hash"] != fingerprint
                or p["expires_at"] is None or p["expires_at"] <= now):
            raise Rejected("approval invalid")
        if ticket["version"] != p["expected_version"]:
            raise Rejected("ticket changed")

        cursor = db.execute(
            "INSERT INTO note (proposal, fingerprint, body) VALUES (?, ?, ?)",
            (proposal_id, fingerprint, p["note"]),
        )
        db.execute(
            "UPDATE ticket SET version = version + 1 "
            "WHERE tenant = ? AND id = ?",
            (tenant, p["ticket"]),
        )
        return cursor.lastrowid

Keep the unique constraint in the database. PostgreSQL offers row-level locking and unique constraints for the same design. For a first write, protect the proposal and ticket from changes between validation and commit, and use a consistent lock order or a conditional update that checks the expected version. Commit the note and state update in one transaction. When the action also publishes a message or calls another system, use an outbox for the event and make its consumer handle duplicates. 7 8 9

Wire the apply route in FastAPI

Save the write handler as boundary.py. This adapter gives the authenticated requester an apply endpoint. Inject three application dependencies. verify_token returns an actor after validating the bearer credential, open_db opens a connection configured with sqlite3.Row, and clock returns the server time in the same units as the approval expiration. In production, use your identity provider and an authorization lookup to bind tenant membership. If you use JWT access tokens, validate the signature, accepted algorithm, issuer, audience, and time claims according to that provider’s contract. FastAPI extracts credentials, and the verifier determines whether to trust them. 17 18

The route receives a proposal identifier. It does not read identity or time from the request body. A missing proposal and a proposal the requester cannot access share the same response. An authenticated owner can see a conflict for a pending, stale, or changed proposal. The verified approval record is created by the separate review operation.

from contextlib import closing
from dataclasses import dataclass
from typing import Annotated

from fastapi import Depends, FastAPI, HTTPException
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer

from boundary import Rejected, apply_note


@dataclass(frozen=True)
class Actor:
    tenant: str
    user: str


def make_api(verify_token, open_db, clock):
    app = FastAPI()
    bearer = HTTPBearer(auto_error=False)

    def signed_in(
        auth: Annotated[
            HTTPAuthorizationCredentials | None, Depends(bearer)
        ],
    ) -> Actor:
        actor = verify_token(auth.credentials) if auth else None
        if actor is None:
            raise HTTPException(
                status_code=401,
                detail="Sign in to continue.",
                headers={"WWW-Authenticate": "Bearer"},
            )
        return actor

    @app.post("/proposals/{proposal_id}/apply")
    def apply(
        proposal_id: str, actor: Annotated[Actor, Depends(signed_in)]
    ):
        with closing(open_db()) as db:
            try:
                receipt = apply_note(
                    db, actor.tenant, actor.user, proposal_id, clock()
                )
            except Rejected as exc:
                code = str(exc)
                if code in ("proposal unavailable", "ticket unavailable"):
                    raise HTTPException(404, "Proposal unavailable.") from exc
                raise HTTPException(409, code) from exc
        return {"status": "applied", "receipt": receipt}

    return app

Uvik Software ran six HTTP fixture tests against this route using FastAPI 0.141.1, Python 3.12.14, and SQLite 3.53.1. The tests use an injected token lookup, a temporary SQLite file, and a seeded approval. All six passed, including two requests submitted concurrently through separate test clients. Both received the same receipt, and the database held one note and one ticket version increment. Exercise the review endpoint, real identity provider, and production database in your integration environment. 11

Verify the resulting state

Verify the committed note, its proposal and ticket, and the resulting ticket version. A model message and a tool trace can explain what the agent attempted. The persisted records show what happened. Anthropic’s guidance on agent evaluations distinguishes the trial transcript from the final state in its environment. 10

Uvik Software ran 12 checks using Python 3.12.14 and SQLite 3.53.1. All passed. The accepted case returned the same receipt on replay, with one note and one version increment. The blocked first-write cases left zero notes. A simulated failure during the ticket update rolled back the note insertion. These are observations from the local fixture.

Uvik Software support-ticket write fixture, run 11 November 2026. Each check starts with its own in-memory database.

Put deterministic handler tests in CI. Add an API layer test that supplies an authenticated user, sends an approval decision, and replays the request. Run concurrent requests against the actual production database engine in a test environment. Include a revoked reviewer, a stale version, and ticket text that asks the model to read a different customer’s ticket. Check the tool arguments, authorization decision, and final rows. FastAPI’s TestClient can exercise the HTTP boundary. The OpenAI Agents SDK has test utilities for scripted model responses. 11 12

Keep a separate live-model evaluation set for varied ticket text, ambiguous requests, and timeouts. Reuse the same expected outcomes when comparing changes to prompts, models, or tool descriptions. Repeat cases where model output can vary. A failure to suggest a useful note belongs in the quality results. Any unauthorized write belongs in the release-blocking tests. 10

Follow one proposal through failure and recovery

The following sequence came from the local HTTP fixture. The fixture began with ticket version 7 and an unapproved proposal. It applied the same proposal ID throughout.

Uvik Software FastAPI and SQLite fixture. The approval and ownership changes are controlled test steps.

Inspect the point of failure before deciding what to retry. The 409 for a pending proposal belongs in the review queue. A stale revision needs a fresh draft. A timeout with an uncertain outcome can be retried using the same proposal ID. A permission failure needs an access decision. Log the reason for the operator without revealing inaccessible ticket or proposal details to the requester.

Build a live model test set

Prepare a small set of redacted or synthetic threads with expected actions and outcomes. As a starting design, use 40 cases with 10 each for straightforward follow-ups, incomplete or contradictory information, hostile text inside tickets, and stale or repeated requests. Define which cases should result in a proposal, which should go to a human, and what database state is acceptable. Hold some cases back when tuning prompts. Keep a trace and reference outcome for each case. 10

Grade the note against the source evidence and score the action using the stored state. On a sample, have a support specialist check that the draft preserves uncertainty and gives the next person a useful action. Run each case three times when comparing model or prompt versions. Report both passing trials over total trials and the number of cases that passed every repeat. The second measure shows which workflows are reliably handled on repeated attempts. Include the run count and version in the report. 10

If the queue serves several languages, keep those results separate. The November 2026 τ-bench-LILT preprint evaluated eight agent models and reported drops of up to 61 points relative to English on the all-four-trials success metric, with the largest drops in Korean and Farsi retail. For a multilingual rollout, build a separate eval slice with local terms, policies, and data for each language you plan to support. 19

Define what a small pilot can establish

Freeze the eligible queue, ticket strata, reference outcomes and escalation rules before comparing a model with the baseline. For example, the current human process writes an accurate note, defers or closes every eligible ticket. Report the model variant against the same three outcomes. Have a reviewer blind to the variant adjudicate a sample, with extra review where adjudicators disagree. Preserve attempts that time out or never produce a proposal in the denominator. Keep a red-team set separate from the traffic sample used to estimate ordinary performance.

Use the test layers to answer different questions. A deterministic test verifies a known rejection or commit. A replay with a scripted model response verifies the application path after a particular proposal. A live test tests whether the model produces a useful proposal or defers when it should. A monitored deployment is the stage that measures queue effect and real reviewer time.

Set a hard stop on an observed unauthorized write or duplicate effect. Test the controls that prevent both directly. When estimating a rate from a pilot with zero incidents, an exact one-sided 95% binomial upper bound is one minus 0.05 raised to the power of one divided by the sample size. With 300 independent, representative opportunities at a stable rate, zero events still leaves an upper bound of about 1.0%. Repeated versions of the same ticket and the 20 deliberately selected cases above serve different purposes. 26

Reproduce the approval and replay checks

Copy the handler above into check_note.py. Append the setup below and run python3 check_note.py. It checks a pending proposal, an approved write, and a replay of the same request. The note remains linked to its proposal and the ticket version advances once.

SCHEMA = """
CREATE TABLE ticket (
    tenant TEXT NOT NULL, id TEXT NOT NULL, owner TEXT NOT NULL,
    version INTEGER NOT NULL, PRIMARY KEY (tenant, id)
);
CREATE TABLE reviewer (
    tenant TEXT NOT NULL, id TEXT NOT NULL, active INTEGER NOT NULL,
    PRIMARY KEY (tenant, id)
);
CREATE TABLE proposal (
    id TEXT PRIMARY KEY, tenant TEXT NOT NULL, ticket TEXT NOT NULL,
    requester TEXT NOT NULL, expected_version INTEGER NOT NULL,
    note TEXT NOT NULL CHECK (length(trim(note)) BETWEEN 1 AND 500),
    approved_by TEXT, approved_hash TEXT, expires_at INTEGER
);
CREATE TABLE note (
    id INTEGER PRIMARY KEY, proposal TEXT UNIQUE NOT NULL,
    fingerprint TEXT NOT NULL, body TEXT NOT NULL
);
INSERT INTO ticket VALUES ('acme', 'T-42', 'sam', 7);
INSERT INTO reviewer VALUES ('acme', 'riley', 1);
INSERT INTO reviewer VALUES ('acme', 'sam', 1);
INSERT INTO proposal VALUES (
    'p-1', 'acme', 'T-42', 'sam', 7, 'Device logs requested.',
    NULL, NULL, NULL
);
"""


db = sqlite3.connect(":memory:")
db.row_factory = sqlite3.Row
db.executescript(SCHEMA)
try:
    apply_note(db, "acme", "sam", "p-1", 100)
except Rejected:
    assert db.execute("SELECT count(*) FROM note").fetchone()[0] == 0
else:
    raise AssertionError("pending proposal wrote a note")

p = db.execute("SELECT * FROM proposal WHERE id = 'p-1'").fetchone()
with db:
    db.execute(
        "UPDATE proposal SET approved_by = 'riley', approved_hash = ?, "
        "expires_at = 200 WHERE id = 'p-1'",
        (proposal_hash(p),),
    )
receipt = apply_note(db, "acme", "sam", "p-1", 100)
assert apply_note(db, "acme", "sam", "p-1", 100) == receipt
assert db.execute("SELECT count(*) FROM note").fetchone()[0] == 1
stored = db.execute("SELECT proposal, body FROM note").fetchone()
assert tuple(stored) == ("p-1", "Device logs requested.")
assert db.execute("SELECT version FROM ticket").fetchone()[0] == 8
print("Pending proposal blocked; approved replay returned one receipt.")
db.close()

Trace what happened and release in stages

Use one run ID to correlate the model run, tool proposal, reviewer decision, database write, and receipt. Record the model and configuration version, tool name, policy decision, latency, and outcome. Separate proposal, approval, and commit counts. A high proposal count with few useful approvals can reveal a quality issue even when the system writes reliably.

Use a field allowlist for logs. Ticket content, personal data, and raw tool arguments should stay out of routine telemetry unless there is a documented purpose and suitable access and retention controls. The OpenAI Agents SDK collects model and tool input and output in traces by default. Set trace_include_sensitive_data=False for that run and configure custom spans with the same care. 13

Start in suggestion mode with a small support group. Compare proposed notes with what reviewers actually choose to write. Enable the commit route for that group when your state and permission tests pass and there is a clear escalation owner. Keep a service flag that can stop new writes while preserving decisions and audit records. Investigate a blocked write through the support workflow and preserve the proposal so the team can see what happened.

Calculate cost per valid action

Set the cost window to the same set of eligible tickets used for the quality report. Include model calls, tool calls, retries, allocated application costs, and the time reviewers spent accepting and rejecting drafts. A valid action is a unique committed note that satisfies the ticket criteria. A matching receipt on replay confirms the same action. Count it once. Track successful handoffs separately so a variant cannot appear cheaper by writing only to the easiest tickets.

The example compares two configurations over the same-sized sets of 120 eligible tickets. All values below are illustrative. Reviewer labor is valued at $45 an hour.

Divide $38.55 by 57 for configuration A and $38.00 by 65 for B. In this example, the configuration with higher model and tool spend costs less per valid action because it produces more valid actions and uses fewer reviewer minutes. Both columns exclude remaining manual support work. Include that work when making a business case for the whole queue.

Set rollout gates the team can operate

Use a versioned record of the prompt, model, tool schemas, and application deployment for every run. Record whether the ticket entered shadow mode, needed a reviewer, or reached a committed state. Decide the thresholds before the pilot and use the same denominators as traffic changes. Google’s SRE guidance recommends evaluating a limited canary against the remainder of the service before expanding. 20 21

For the canary dashboard, count blocked attempts per write request, stale proposals per first-time apply attempt, and duplicate effects per repeated proposal. Measure draft latency from ticket acceptance to proposal or handoff. Keep reviewer queue age separate from model response latency so the team can identify the source of delay. If approvals accumulate, keep the suggestion route available while slowing or stopping new writes.

Give the on-call owner a way to disable new writes for a tenant and inspect receipts for pending operations. Keep the original proposal, reviewer decision, and committed receipt linked for investigation. If a note needs correction, use the support system’s audited correction flow and capture the failure as a new eval case.

The first production decision

Start with the narrowest action for which the team can establish the facts, authority and final effect. Keep the rule in production if it handles the eligible cases with less review. Use a bounded model to recover useful coverage where interpretation matters. Expand only when the same release record can show how often the model helps, which cases it defers, and whether the application performed only the approved action.

When to bring in a Python engineering team

A prototype can show whether a model drafts a useful note. A production release also needs somebody to own authorization, the ticket system integration, CI, and recovery. Uvik Software’s guarded tool calling case study describes its work on action validation, escalation routing, and regression tests for customer service agents.

If your team already has an agent prototype, bring one proposed action, the system it would change, and a failed or ambiguous example to Uvik Software’s AI agent development team. A senior Python engineer can help define the write boundary and acceptance tests for that first production workflow. Teams with an established integration can focus directly on evaluation and observability.

How to cite this guide

Uvik Software. How to Build a Production AI Agent in Python. 2026. https://uvik.net/blog/ai-agent-development-guide/

Suggested attribution. In Uvik Software’s 20-case scripted ticket-note fixture, a check against current ticket evidence stopped 15 invalid proposals that passed shape validation and preserved all five valid proposals. The guide separately tests approved writes and replay in Python and SQLite.

Sources

1. OWASP GenAI Security Project, Top 10 for Agentic Applications. Tool misuse and identity risk.
2. OpenAI Agents SDK, Tools. Function tools and typed schemas.
3. OpenAI API, Guardrails and Human Review. Tool scope, approval, and application policy.
4. FastAPI, Request Body. Typed and validated request bodies.
5. OpenAI Agents SDK, Context Management. Local context and model-visible data.
6. OpenAI Agents SDK, Human in the Loop. Pausing and resuming approval-gated calls.
7. PostgreSQL, Explicit Locking. Row-level locking during transactions.
8. PostgreSQL, INSERT. Unique conflict handling.
9. AWS Prescriptive Guidance, Transactional Outbox Pattern. Coordinating writes and event delivery.
10. Anthropic, Demystifying Evals for AI Agents. Transcripts, outcomes, and repeated trials.
11. FastAPI, Testing. HTTP tests with TestClient.
12. OpenAI Agents SDK, Testing. Deterministic model and tool interaction tests.
13. OpenAI Agents SDK, Tracing. Trace data and sensitive-field configuration.
14. Barres and colleagues, τ²-Bench, ICML 2026. Tool use, task outcomes, and coordination with a simulated user.
15. Gupta and colleagues, Building Customer Support AI Agents at 100M-User Scale. June 2026 preprint reporting Nubank’s card-delivery agent A/B outcomes.
16. Pydantic, Configuration. Model handling of unexpected fields.
17. FastAPI, Security Tools. HTTP bearer authentication dependency.
18. IETF, JSON Web Token Best Current Practices. Algorithm, issuer, subject, and audience validation.
19. Uhlig and colleagues, τ-bench-LILT. November 2026 multilingual tool-use preprint and repeat-success results.
20. Google SRE Workbook, Canarying Releases. Limited deployment and canary evaluation.
21. Google SRE Workbook, Implementing SLOs. Service level indicators and user-facing outcome measures.
22. Pan and colleagues, Measuring Agents in Production, ICML 2026. Case studies and reports on production or pilot systems. Full methodology and limitations.
23. Alcobaça and colleagues, Screen Before You Serve, 2026 preprint version 2. Tables 2 and 3 report Nubank’s live comparisons and offline screening.
24. Debenedetti and colleagues, AgentDojo. Tool-output prompt injection and task utility benchmark.
25. OpenAI, The Hugging Face Incident and the Road Ahead. Incident scope, containment and investigation.
26. Hanley and Lippman-Hand, Interpreting Zero Numerators, JAMA 1983. Statistical interpretation of zero observed events.