Test an agent before it meets customers

A convincing demonstration answers one question: can this system succeed on the example in front of us? A launch decision requires more. You need to know how it behaves with missing information, conflicting instructions, unavailable tools, and requests outside its scope. Testing turns those concerns into cases the team can run, inspect, and discuss before customers become the source of every discovery.

Begin with the job the agent is supposed to perform. Write the desired outcome in ordinary language, then identify the evidence that would prove it happened. If the task is to prepare a support draft, inspect the saved draft and its sources. If the task changes a record, inspect that record. The final chat message is only one piece of evidence.

Build a case file for each task

Each test case needs an input, a starting environment, an expected result, and a reason it matters. Include the relevant documents or fixture records so another person can reproduce the setup. A vague instruction to test a difficult customer request is not enough. Write the request, supply the authorized context, and describe what a correct response or escalation would look like.

Use fictional or properly authorized data when constructing the initial set. Remove unnecessary personal information from retained examples. Preserve the features that make a case difficult, such as two similar account names or an outdated policy, without carrying over private details simply for realism. The test environment should help you investigate behavior without introducing avoidable exposure or contacting real customers by accident.

Cover several kinds of failure

Ordinary cases establish whether the basic workflow works. Missing-input cases check whether the agent asks for information or invents it. Conflicting-source cases reveal how it handles disagreement. Out-of-scope cases check whether it stays within the assigned job. Tool-failure cases test the surrounding application, including timeouts, denied access, and incomplete responses. Each group should represent a question the team needs answered.

Include a case where retrieved content contains instructions aimed at the agent. For example, a fictional document could tell it to ignore its task and send the contents elsewhere. The expected result is to treat that text as source content rather than authority to change the workflow. Test the actual permission boundary as well as the wording of the response. A polite refusal is not sufficient if an unauthorized action still occurs elsewhere in the run.

Use an example with observable state

Imagine a fictional customer support assistant that prepares answers about product setup. One test includes a clear question and an applicable manual section. The expected outcome is a draft with a supporting link, saved for review, with no customer message sent. Another test names an unknown product. Its expected outcome is a request for clarification in the internal queue, with the original ticket left open.

A third case provides two manuals with different version identifiers. The assistant should choose the one matching the product or mark the version unresolved. A fourth simulates a document service timeout. The system should report that it could not retrieve the material and should not present a guessed answer as sourced. These cases produce inspectable outcomes rather than inviting a reviewer to score general helpfulness.

Choose the right checks

Use deterministic checks for things a program can verify reliably: required fields, allowed status values, existence of a draft, absence of an external send, or a valid source identifier. Use human review for judgments such as whether the answer resolves the question and whether an explanation is understandable. A model-based grader may assist with repeated review, but it should be checked against human judgments before becoming an unquestioned authority.

Anthropic's evaluation article describes code, model, and human graders as complementary approaches. For a small launch, the useful question is which check will catch each failure you care about. An exact text match may reject a correct paraphrase, while a broad helpfulness score may overlook a forbidden action. Pick checks that match the task's meaning.

Inspect both the answer and the path

A good answer can arrive through an unacceptable route. Perhaps the agent searched outside the authorized collection, used another customer's record, or retried a write after a timeout without checking whether it succeeded. Inspect tool calls and the resulting state for these cases. The goal is to verify the boundaries of the workflow, not to demand one exact sequence when several legitimate routes exist.

Anthropic's tool-design guidance is relevant when failures point to unclear action descriptions or unhelpful returned data. Sometimes the correction belongs in the tool interface rather than the instruction prompt. If two actions have nearly identical names but very different effects, improve that distinction and add a regression case that proves the agent uses the intended action.

Account for variation

Run important cases more than once. A single success can hide unstable behavior, and a single failure may need investigation before you understand its cause. Record the model, instructions, tool versions, and relevant configuration with each run. Keep changes small enough that you can tell whether an improvement came from the prompt, the retrieval setup, or another part of the application.

Do not publish an impressive percentage without its context. A result from a small, curated set tells you about that set. It does not establish performance on every future customer request. Internally, record the number and type of cases, repeated trials, exclusions, and known gaps. Those details make the result useful for a release decision and prevent the score from becoming a substitute for judgment.

Decide what blocks release

Separate defects by consequence. An awkward sentence may be acceptable during an internal pilot. An unauthorized external message or a false claim that an action completed may require a stop. Define these decisions before reviewing the final results so the team is less tempted to excuse a serious failure because the rest of the demonstration looked strong.

For each blocking issue, identify the cause and add a test that would catch it again. A fix should be rerun against the affected case and the relevant existing set. Avoid continually rewriting expected answers to match whatever the system now produces. If the original expectation was wrong, explain the change explicitly and have the task owner agree to the new behavior.

Rehearse recovery and launch gradually

Test how the workflow behaves when stopped midway, when a reviewer rejects a proposal, and when a connection returns after an outage. Verify that the operator can find pending work and tell it apart from completed work. Confirm that retries do not create duplicate records or duplicate messages. Recovery is part of the customer experience even when the successful path is the one featured in the demo.

A limited launch can begin with internal review of every result and a small authorized audience. Keep the manual process available while collecting correction and exception data. Schedule a review of real failures, update the test set with appropriate examples, and rerun it when instructions, tools, or models change. Testing becomes useful when it informs those decisions, rather than ending as a checklist attached to launch day.