How to backtest an AI agent without future-data leakage

What future-data leakage looks like

Leakage happens when a historical run receives information created after its decision time. A page may carry an old publication date but include a later correction. A customer record may show the current plan rather than the plan held during the support case. Search rankings, embeddings, summaries, model weights, and internal policies can all contain later knowledge even when the prompt itself looks historical.

The rule is simple: if the production agent could not have read a value at the cutoff, the replay cannot read it either. Scikit-learn's TimeSeriesSplit documentation explains why ordinary cross-validation is unsuitable for ordered data: it can train on the future and evaluate on the past. Agent backtests need the same chronological discipline, applied to every tool and source.

Freeze six parts of the run

A seven-step backtesting workflow

1. Define the decision and observation window. “Escalate this account at 09:00” can be scored after seven days. “Will this company be acquired?” needs a longer fixed horizon. Do not let each case choose its own convenient window.
2. Build an append-only case ledger. Store the cutoff, source versions, expected action, later outcome, and who reviewed the label. Keep the outcome in a separate field that the runner cannot load.
3. Reconstruct every tool response. Replay database queries against temporal tables or snapshots. For web research, store fetched content and retrieval rank at the cutoff. A current web search is not historical evidence.
4. Choose chronological folds. Tune prompts on earlier cases, test on the next block, then move the cutoff forward. Add a gap when the label window overlaps the following fold.
5. Run a simple baseline. Compare with a fixed rule, keyword filter, or previous human process. An agent that beats an older prompt but loses to a two-line rule has not earned production complexity.
6. Score decisions and evidence separately. Record task success, false positives, false negatives, abstentions, unsupported claims, tool failures, tokens, elapsed time, and human review time.
7. Reserve an untouched final period. Use it once after prompt and tool choices stop changing. If you tune after reading it, create a newer final period.

Anthropic's evaluation guide shows exact-match and model-graded examples, while NIST describes test, evaluation, verification, and validation as work that continues through design, deployment, and operations. A backtest supplies evidence before release, but production monitoring still has to catch new conditions.

Worked example: an account-risk agent

Suppose an agent reviews SaaS accounts each Monday and recommends “contact,” “wait,” or “close.” The label is whether the account cancels within 30 days. For a case dated March 2, the agent may read invoices, support messages, product usage, and plan data recorded by March 2. It may not read the March 9 cancellation request, a later support summary, or today's customer status.

Create monthly folds. Use January and February to revise the prompt, leave a seven-day gap so late-arriving events cannot cross the boundary, and test on March. Move forward until the prompt is fixed, then run April once as the untouched period. Compare the agent with a rule that contacts any account with two failed payments or a 50% usage drop. Score cancellation recall, unnecessary contacts, abstentions, source support, cost, and reviewer minutes.

This example exposes an easy mistake: a current customer table may overwrite historical plan and status fields. The query must use temporal rows or a saved snapshot. Recording only the old prompt while reading today's database produces a polished replay with invalid evidence.

Backtest, shadow test, or live experiment?

• Use a backtest when you have reliable historical inputs and outcomes, and when mistakes are too costly for an early live trial.
• Use a shadow test when historical evidence cannot be reconstructed. Let the agent process current cases without acting, then compare its recommendations with later outcomes and human decisions.
• Use a limited live experiment only after offline and shadow results pass, with bounded actions, approval rules, monitoring, and a rollback path.
• Use a prospective benchmark for changing web questions. Freeze questions and predictions now, then score them after the outcome window closes.

Limits and common mistakes

• A historical search index may not exist. Saved page copies help, but they do not reproduce the ranking that shaped the original evidence.
• A newer model may contain knowledge from after the cutoff. Tool isolation prevents retrieved leakage, but it cannot remove facts already learned during training.
• Repeated prompt changes overfit a small backtest. Keep an untouched period and report how many variants you tried.
• Outcome labels can arrive late or reflect human intervention. Define when a label becomes final and record actions that changed the result.
• One average score can hide harmful groups or costly false positives. Break results down by decision type, risk, and user segment where lawful and appropriate.
• Historical performance does not prove current safety. Data, users, policies, tools, and models change after release.