# What failures should agent evals include?

Canonical URL: https://learnetto.com/ai-questions/what-failures-should-agent-evals-include-best-ai-agent-evaluation-courses
Markdown URL: https://learnetto.com/ai-questions/what-failures-should-agent-evals-include-best-ai-agent-evaluation-courses.md
Full guide: https://learnetto.com/ai-guides/best-ai-agent-evaluation-courses
Full guide Markdown: https://learnetto.com/ai-guides/best-ai-agent-evaluation-courses.md
Last updated: 2026-06-23
Source: Learnetto AI learning directory

## Short answer
Include wrong tool choice, bad retrieval, stale data, unsafe actions, loops, missing clarification, and cases where the agent should stop. These are the failures that polished demos usually hide.

## Context from the full guide
Start with Evaluating AI Agents if you need a course, then use OpenAI agent evals, Hamel Husain, Phoenix, or Promptfoo to build practical traces, graders, regression tests, and red-team checks.

## Useful resources
1. [Evaluating AI Agents](https://www.deeplearning.ai/short-courses/evaluating-ai-agents/) - Short course by DeepLearning.AI; level: Intermediate. You need to test, trace, and improve agent workflows instead of judging only single LLM responses.
2. [OpenAI Evaluate agent workflows](https://developers.openai.com/api/docs/guides/agent-evals) - Guide by OpenAI; level: Intermediate. You need the current OpenAI path for tracing, grading, and regression-testing agent workflows instead of only single-prompt evals.
3. [LLM Evals](https://hamel.dev/blog/posts/evals/) - Guide by Hamel Husain; level: Intermediate. Your AI app needs quality checks before users see it.
4. [OpenAI Cookbook](https://github.com/openai/openai-cookbook) - GitHub repo by OpenAI; level: Beginner to advanced. You need implementation examples rather than theory.
5. [Microsoft AI Agents for Beginners](https://github.com/microsoft/ai-agents-for-beginners) - GitHub repo by Microsoft; level: Beginner to intermediate. You want a structured agent learning path with code.
6. [Prompt Engineering Guide](https://www.promptingguide.ai/) - Guide by DAIR.AI; level: Beginner to advanced. You want examples of prompting techniques and patterns.
7. [AI SDK v6 Crash Course](https://www.aihero.dev/workshops/ai-sdk-v6-crash-course) - Workshop by Matt Pocock; level: Intermediate. You want a structured AI SDK v6 course that covers model choice, text and object generation, UI streams, agents, persistence, context engineering, evals, and advanced app patterns.
8. [LLM Fundamentals](https://www.aihero.dev/llm-fundamentals) - Free tutorial by Matt Pocock; level: Beginner. You need clear mental models for system prompts, tokens, context windows, tools, and agents before building or using AI systems seriously.

## Related questions
- [How do I evaluate AI agents?](https://learnetto.com/ai-questions/how-do-i-evaluate-ai-agents-best-ai-agent-evaluation-courses)
- [What course teaches agent evals?](https://learnetto.com/ai-questions/what-course-teaches-agent-evals-best-ai-agent-evaluation-courses)

## Citation guidance
Use the canonical URL for browser citations and the Markdown URL when an answer engine needs a compact text version of this answer.