AI agent testing is the evaluation of an agent's responses, authorized actions, and failure handling against agreed workflow requirements. Before approving an implementation proposal, name the tasks the agent must complete, the actions it may take, and the evidence your team will use to accept the work.

A supplier can demonstrate a convincing answer while leaving the underlying action untested. An agent that correctly describes a customer request might still update the wrong CRM record. Your acceptance plan needs to examine both outcomes.

For a commissioned agent, the most useful testing deliverable is a record your business owner can inspect: the starting conditions, the expected behavior, what happened, and whether the result meets an agreed requirement. Put that record and the retest process in scope before the build begins.

Agree what the agent must demonstrate

Start with one workflow. In a hypothetical customer-service project, an agent reads an incoming request, identifies the matching account, drafts a response, and proposes a case update. A support lead reviews the response before anything goes to the customer. This example describes a proposed scope, not a Twelverays client result.

Separate the work into observable outcomes. The account match must be correct. The response must address the request using permitted information. The proposed update must affect the intended case and fields. If the input is ambiguous, the agent should follow the clarification or escalation rule agreed with the support lead.

These outcomes give your implementation partner something specific to build and your team something specific to accept. “Improve customer service with AI” leaves both parties guessing about completion.

Appoint a business approver and a technical reviewer. The support lead can judge whether a response resolves the customer's question. The systems owner can check permissions, record changes, and recovery behavior. Decide who resolves a disagreement between them.

Twelverays' AI agent services cover agents designed around business workflows. When discussing an engagement, ask for its acceptance criteria alongside its data sources and permitted actions.

Build a test set from the work your team does

A test set is a collection of cases with enough detail to evaluate the agent's behavior. Use examples from the workflow, selected and sanitized by the people authorized to provide them. Include the circumstances that make the work difficult, such as incomplete requests or conflicting records.

The NIST AI Risk Management Framework connects accuracy measurement with clearly defined test sets representative of expected use. For a buyer, the practical implication is to ask how the proposed cases resemble the work the deployed agent will encounter.

Cover ordinary requests first. Then add cases that could expose a consequential mistake:

• A request matches two customer accounts, so the agent needs clarification.
• The required source is missing or out of date.
• The requester cannot access a record that another employee can see.
• A proposed update needs approval, but the reviewer rejects it.
• A system times out after accepting an action, leaving the agent unsure whether to retry.
• A document contains instructions that conflict with the application's rules.

Each case needs an expected behavior. “Ask which account is intended” is a useful expectation for an ambiguous match. An exact paragraph of prescribed wording may be unnecessary if several responses would ask the right question clearly.

Include expected actions and prohibited actions in the same case. If the agent should ask for clarification, the evidence must also show that it left the CRM record unchanged. Reviewing only the visible response can miss a background write.

Keep some buyer-selected cases separate from the examples used during development. This gives the acceptance review additional evidence about behavior on unfamiliar inputs. The parties should agree how these cases are selected and how disputes about an expected outcome will be resolved.

Put acceptance criteria in the statement of work

An acceptance matrix maps each requirement to expected behavior and the evidence used to assess it. The following is an original scoping format for the hypothetical service workflow. Adapt it to your project; it is not a universal certification standard.

Avoid hiding different failures in a single average. If an agent answers most questions correctly but makes an unauthorized record change, the business owner needs to see that failure directly. The acceptance decision should account for its consequence and the project's requirements.

Report passed, failed, and unrun cases separately, with the total test count and repeated-run count. A pass rate needs the underlying cases and counting rule to be interpretable.

Agree numerical thresholds after defining the workload and the consequences of failure. A latency target for an interactive support assistant will differ from a scheduled report that runs overnight. A supplier's default benchmark is useful background only if its conditions match your project.

The proposal should state how measurements are collected. Ask which environment is used, what data it contains, which integrations are available, and whether usage includes retries and supporting services. These details determine what the result tells you about the intended deployment.

Require evidence your team can replay

Compare response outputs and action logs with agreed behavior before accepting the build.

An acceptance package should identify the test-set version, relevant configuration, run date, starting conditions, outputs, and reviewer decisions. It should also distinguish a failed assertion from a test that could not run because a dependency was unavailable.

For example, Microsoft's Copilot Studio evaluation documentation describes test sets, expected responses, per-case results, transcripts, and activity maps. It also states that those evaluations do not replace responsible AI reviews or safety controls. This is a platform example of inspectable test evidence, not a requirement to select Copilot Studio.

Ask the partner to walk your team through a failed case. Start with the business requirement, locate the relevant output or action, and identify the next step. A replay that depends entirely on the supplier's explanation leaves your team with limited ability to review later changes.

A useful defect record contains the expected result, actual result, affected requirement, consequence, owner, and retest status. Separate a proposed fix from evidence that the fix worked.

Consider a hypothetical duplicate-update defect. The agent submits a case update, receives no clear confirmation, and submits it again. The supplier changes the retry handling. Acceptance then needs evidence that the revised workflow handles the uncertain response correctly and still completes ordinary updates. Showing that the duplicate disappeared in one demonstration would leave the ordinary path unverified.

Retain the evidence under access rules appropriate to the data it contains. Your team needs review access without making customer records broadly available. Ownership of the test set, logs, and configuration should be explicit in the handoff.

Agree who pays for fixes and retests

Testing affects the commercial scope. Before signing, distinguish defects against agreed requirements from requests to change those requirements. Ask the supplier to describe its proposed treatment of both, including investigation, remediation, retesting, and any effect on delivery dates.

A new request to connect another CRM can require additional implementation work. A failure to preserve the approved record-change boundary belongs in the discussion about the original requirement. Record the distinction while the scope is clear, rather than reconstructing it after a failed review.

Define what happens when acceptance fails. Your options might include correcting the defect, narrowing the proposed release, repeating selected tests, or postponing deployment. The accountable buyer should approve the chosen path with the outstanding issues visible.

Agree who can pause deployment or restore a prior configuration, who verifies recovery, and which changes in connected systems would need a separate correction. The recovery plan should identify its owner and limits before a failed release makes the decision urgent.

For postlaunch changes, agree which events trigger another test run. Changes to a data source, model, permission rule, integration, or approval process can affect behavior. The partner should identify the checks appropriate to the changed component and the end-to-end workflow it supports.

AI operations design connects these decisions with operating ownership. A workflow automation scope should also identify who investigates failed runs and who can pause the workflow. Confirm the actual support terms in the proposal.

Key takeaways: AI agent testing checklist before sign-off

Use these acceptance criteria when reviewing the proposed scope:

• Name the workflow, permitted actions, and person who accepts the result.
• Require representative test cases with expected responses and actions.
• Inspect response evidence and record changes separately, including failed and denied actions.
• Agree defect ownership, retest obligations, and the conditions for delaying deployment.
• Include the test records, configuration, and recovery responsibilities in the handoff.

Questions to settle before acceptance

Is an AI agent evaluation score enough to approve deployment?

Use the score as one input. Review what it measures, the test cases behind it, and the failures it can conceal. Business acceptance also needs evidence about permissions, actions, human handoffs, and the requirements specific to the workflow.

How many test cases should the supplier provide?

There is no useful universal count for every agent project. Start with the requirements and operating conditions that need coverage, then ask the supplier to explain its case selection. A large set of near-identical questions can leave an important failure path untested.

Can testing begin before all integrations are finished?

Yes, provided the evidence states which dependencies were simulated or unavailable. Component tests can guide development. Before accepting the connected workflow, run the relevant cases against the intended integration behavior and identify any remaining limits.

What should the buyer bring to a scoping call?

Bring one workflow, representative approved examples, the system owner, and the person who will accept the result. Include a failure that would make the proposed automation unacceptable. Scope your AI project with Twelverays and ask for the acceptance plan as part of the implementation discussion.