AI directory search

Search across educators, skills, and resources.

Use this when you know the topic you need: Claude Code, MCP, evals, RAG, agents, product, coding, prompting, foundations, or model internals.

52 matches for "evaluation"

Video matches

Watch first when you want a fast feel for the topic before opening courses, docs, or profiles.

Gemini 4 Hydrogen: Price, Access & Should You Switch? (Nov 2026 Guide) video thumbnail ►

Gemini 4 Hydrogen: Price, Access & Should You Switch? (Nov 2026 Guide)

AI Inbox Academy · Gemini 4 Hydrogen, Gemini 3.8 Flash, model migration, coding agents

Claude Fable 5.2: What the Benchmarks Actually Show video thumbnail ►

Claude Fable 5.2: What the Benchmarks Actually Show

AI Mathias | AI Automation · Claude Fable 5.2, Claude Opus 5.5, coding benchmarks, FrontierCode 2.0

LifeVal: Measuring AI progress on the tasks that matter in everyday life. video thumbnail ►

LifeVal: Measuring AI progress on the tasks that matter in everyday life.

OpenAI · LifeVal, AI agents, personal assistants, evaluation

Anthropic Opened Its Books. What Is It Really Worth? video thumbnail ►

Anthropic Opened Its Books. What Is It Really Worth?

Patrick Boyle · 2026, Anthropic, IPO, S-1

How to Build RAG From Scratch in 50 Lines of Python - Full Tutorial video thumbnail ►

How to Build RAG From Scratch in 50 Lines of Python - Full Tutorial

Tech With Tim · 2026, RAG, Python, Ollama

Scaling Inference-Time Compute with Teams of Agents for Software Engineering video thumbnail ►

Scaling Inference-Time Compute with Teams of Agents for Software Engineering

Open Data Science and AI Conference · 2026, multi-agent systems, software engineering, inference-time compute

OpenAI's GPT-6.1 Terra uses your computer for less video thumbnail ►

OpenAI's GPT-6.1 Terra uses your computer for less

Coding with Sela · 2026, GPT-6.1 Terra, computer use, model selection

Don't Trust Your AI Agents. Test Them Like a Scientist — Stephanie Jarmak, Omni Analytics video thumbnail ►

Don't Trust Your AI Agents. Test Them Like a Scientist — Stephanie Jarmak, Omni Analytics

AI Engineer · 2026, agent evaluation, experimental design, benchmarks

1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life video thumbnail ►

1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life

AI Engineer · 2026, AI incidents, risk, agent evaluation

Educators

Ben Lorica profile photo

Ben Lorica

The Data Exchange · Intermediate

Good practitioner interviews across data, ML, and AI engineering.

Skills

Data systems, ML engineering, AI trends

Greg Kamradt profile photo

Greg Kamradt

Data Independent AI tutorials · Beginner to intermediate

Practical walkthroughs for retrieval, LLM application patterns, and common developer questions.

Skills

RAG, LLM apps, Prompting, Evaluation

Providers and platforms

Weights & Biases profile photo

Weights & Biases

W&B Courses · Intermediate

Good for builders who need to measure, debug, and improve LLM apps rather than just demo them.

Topics

LLM apps, Evals, Experiment tracking, MLOps

Useful for teams building repeatable AI product processes around prompts, datasets, and evaluations.

Topics

Prompt management, Evals, LLM workflows

A strong foundation for people who need the math and modeling basics under applied AI.

Topics

ML foundations, Supervised learning, Unsupervised learning, Model evaluation

OpenRouter profile photo

OpenRouter

OpenRouter docs · Beginner to intermediate

Useful for learning model comparison, latest-family aliases, routing, fallback behavior, agent construction, project-specific evals, and API-compatible experimentation across proprietary and open model families.

Topics

Model routing, Model comparison, Auto Router, Agent SDK, Coding-agent harnesses, GPT models, Claude models, Gemini, Llama, Mistral, DeepSeek, Qwen, API examples, Evaluation

Fireworks AI profile photo

Fireworks AI

Fireworks AI model catalog · Intermediate to advanced

Official material for comparing serverless and dedicated model serving, training specialized models, and measuring quality, token use, cost, and task duration on production-shaped evaluations.

Topics

Ember-1, Kimi K3, Serverless inference, Dedicated deployment, Model training, Model evaluation, Reasoning efficiency

Resources

Gemini 4 Neon launch and evaluation guide

Frontier model news analysis · Google DeepMind · Intermediate to advanced

Evaluate Gemini 4 Neon for frequent coding, analysis or agent workloads before migrating from Hydrogen, Carbon, Argon or Gemini 3 models.

Gemini 4 Neon, coding agents, model routing, Gemini API, model evaluation

GPT-6.2 Astra and Sol launch guide

Frontier model launch analysis · Learnetto · Intermediate

Compare GPT-6.2 Astra and Sol, understand the safety disclosures and run a workload test before changing production models.

OpenAI, GPT-6.2 Astra, GPT-6.2 Sol, Codex, model evaluation

Qwen4-Max launch and evaluation guide

Frontier model analysis · Learnetto · Intermediate

Understand what Alibaba shipped with Qwen4-Max, separate launch claims from independent tests and decide whether to run a controlled agent evaluation.

Qwen4-Max, Alibaba, Qwen, AI agents, coding models

Gemini 4 Hydrogen migration and evaluation guide

Frontier model news analysis · Google DeepMind · Intermediate to advanced

Evaluate Gemini 4 Hydrogen for coding, multimodal analysis or agent workflows, and plan a measured migration from Gemini 3.8 Flash.

Gemini 4 Hydrogen, coding agents, model routing, multimodal AI, Gemini API

Claude Fable 5.2 and Mythos 5.2 launch guide

Frontier model news analysis · Anthropic · Intermediate to advanced

Use this analysis to decide whether Claude Fable 5.2 belongs in a demanding agent workflow, plan migration tests and understand why Mythos access remains restricted.

Claude Fable 5.2, Claude Mythos 5.2, Claude Code, coding agents, model evaluation

Inkling 2 Small release and evaluation guide

Open-weight model release analysis · Thinking Machines Lab · Intermediate to advanced

Use this guide to assess Inkling 2 Small's architecture, planned open weights, benchmarks, costs and fit for your work.

Inkling 2 Small, Thinking Machines Lab, open weights, Mixture of Experts, agentic coding

Mistral Large 4 open-weight release guide

Open-weight model launch analysis · Mistral AI · Intermediate to advanced

Use this to decide whether Mistral Large 4 belongs in a self-hosted coding or defensive-security evaluation.

Mistral Large 4, open weights, coding agents, cybersecurity, self-hosting

Gemini 4 Carbon cyberdefense launch guide

Frontier model and cybersecurity launch analysis · Google DeepMind · Intermediate to advanced

Use this to assess Gemini 4 Carbon's reported cyber capabilities, limited rollout and evidence requirements before planning a production evaluation.

Gemini 4 Carbon, cybersecurity, coding agents, vulnerability research, prompt injection

GPT-6.1 Terra launch explained

Frontier model launch analysis · OpenAI · Intermediate

You need to decide whether GPT-6.1 Terra fits a visual, computer-use or coding workflow and want launch facts separated from independent evidence and unresolved safety questions.

OpenAI, GPT-6.1 Terra, computer use, coding agents, model pricing

Avalon-M1 agent teams and the oversight gap

Multi-agent safety analysis · General Reasoning · Intermediate to advanced

General Reasoning trained agent teams with one shared reward. Performance improved, but simulated reporting to human overseers fell sharply as training continued.

Avalon-M1, RLCR, multi-agent systems, agent safety, human oversight

Reduce coding agent cost without lowering quality

AI coding agent guide · Learnetto · Intermediate

Your coding agent completes useful work but spends too many tokens, repeats retrieval, or carries noisy tool output through later turns.

coding agents, cost, context engineering, tool output, prompt caching

Kimi K3.1 Pro explained

Open-weight model news analysis · Moonshot AI · Intermediate to advanced

Moonshot AI launched Kimi K3.1 Pro on October 24, 2026, with open weights due by November 4. Learn what ships now, how cyber training was restricted and how to test it before migrating.

Kimi K3.1 Pro, Moonshot AI, open weights, coding agents, cybersecurity

Mythos 5 AISI incident: what the new analysis found

AI safety incident analysis · UK AI Security Institute · Intermediate to advanced

UK AISI and Anthropic published new analyses of a cyber test in which Mythos 5 targeted a real open-source maintainer. Learn what happened, what the replay studies found and how to contain long-running agents.

Claude Mythos 5, UK AISI, AI agent safety, supply chain security, agent monitoring

Building and Evaluating Advanced RAG Applications

Short course · DeepLearning.AI · Intermediate

You already know basic RAG and need better retrieval, evaluation, and production-quality patterns.

rag, evals, retrieval, llm apps, ai engineering

Promptfoo

Open-source eval and red-team framework · OpenAI / Promptfoo · Intermediate to advanced

Declarative regression tests and side-by-side comparisons of prompts, models, RAG systems, and agent configurations, especially when red teaming is also required.

evals, llm evaluation, ai quality, open-source frameworks

DeepEval

Pytest-style LLM evaluation framework · Confident AI · Intermediate to advanced

Engineering teams that want LLM and agent evaluations to behave like software unit tests, with thresholds, assertions, and CI-friendly failures.

evals, llm evaluation, ai quality, open-source frameworks

Ragas

RAG and AI-application evaluation library · Vibrant Labs · Intermediate to advanced

Evaluating retrieval quality, grounded generation, agent tool use, and production-aligned test data for RAG applications.

evals, llm evaluation, ai quality, open-source frameworks

Inspect AI

Model and agent evaluation harness · UK AI Security Institute · Intermediate to advanced

Rigorous, reproducible model and agent capability or safety evaluations involving tools, multi-turn interaction, coding, or sandboxed environments.

evals, llm evaluation, ai quality, open-source frameworks

Arize Phoenix

AI observability and evaluation platform · Arize AI · Intermediate to advanced

Teams that want evaluation connected to OpenTelemetry traces, datasets, experiments, prompt iterations, and production troubleshooting.

evals, llm evaluation, ai quality, evaluation platforms

OpenEvals

Reusable evaluator library · LangChain · Intermediate to advanced

Python or TypeScript developers who want composable evaluator functions without adopting a full evaluation platform.

evals, llm evaluation, ai quality, open-source frameworks

Giskard

Agent testing and red-team framework · Giskard AI · Intermediate to advanced

Behavioral tests and adversarial scans of multi-turn agents, chatbots, and RAG systems from a pytest-compatible workflow.

evals, llm evaluation, ai quality, open-source frameworks

Braintrust

AI evaluation and observability platform · Braintrust Data · Intermediate to advanced

Teams that want a feedback loop from production traces and failures into datasets, regression experiments, CI gates, and continuous scoring.

evals, llm evaluation, ai quality, evaluation platforms

LangSmith

Agent evaluation and observability platform · LangChain · Intermediate to advanced

Agent teams, especially LangChain and LangGraph users, that need tracing, datasets, experiments, human review, and production feedback in one system.

evals, llm evaluation, ai quality, evaluation platforms

Langfuse

Open-source evals and observability platform · ClickHouse · Intermediate to advanced

Teams prioritizing open-source data control and one workflow across tracing, prompts, datasets, experiments, annotation, and evaluation.

evals, llm evaluation, ai quality, evaluation platforms

W&B Weave

LLM and agent evaluation platform · Weights & Biases · Intermediate to advanced

Teams already using Weights & Biases or wanting versioned datasets, scorers, traces, and evaluation comparisons in an experiment-oriented workflow.

evals, llm evaluation, ai quality, evaluation platforms

Opik

Open-source agent evaluation platform · Comet · Intermediate to advanced

Teams wanting hosted or self-hosted observability with failure-driven regression suites, experiments, metrics, and human review.

evals, llm evaluation, ai quality, evaluation platforms

MLflow GenAI evaluation

Open-source GenAI evaluation and monitoring · MLflow Project · Intermediate to advanced

Teams extending an existing MLflow or MLOps stack to LLM and agent tracing, evaluation-driven development, human feedback, and monitoring.

evals, llm evaluation, ai quality, evaluation platforms

OpenAI Evals

Hosted LLM and agent evaluation API · OpenAI · Intermediate to advanced

Existing OpenAI API teams maintaining dataset-based prompt or model regression tests during the remaining service window.

evals, llm evaluation, ai quality, cloud and provider evals

Vertex AI Gen AI evaluation service

Managed model and agent evaluation service · Google Cloud · Intermediate to advanced

Vertex AI teams evaluating model outputs, prompts, RAG responses, function calls, and agent trajectories with managed or custom metrics.

evals, llm evaluation, ai quality, cloud and provider evals

Microsoft Foundry evaluations

Cloud evaluation portal and SDK · Microsoft · Intermediate to advanced

Microsoft Foundry teams evaluating models, RAG systems, and single- or multi-turn agents for quality, safety, task completion, and tool use.

evals, llm evaluation, ai quality, cloud and provider evals

Amazon Bedrock Evaluations

Managed model and RAG evaluation jobs · Amazon Web Services · Intermediate to advanced

AWS teams comparing Bedrock models or assessing knowledge bases and external RAG sources with automated metrics, LLM judges, or human reviewers.

evals, llm evaluation, ai quality, cloud and provider evals

Anthropic evaluation guide

Evaluation methodology guide · Anthropic · Intermediate to advanced

Teams designing task-specific Claude and agent evaluation suites with realistic cases, clear success criteria, calibrated graders, and regression gates.

evals, llm evaluation, ai quality, cloud and provider evals

Harbor

Agent evaluation and optimization harness · Harbor Framework Team · Intermediate to advanced

Running coding and computer-use agents against reproducible, sandboxed task suites at local or cloud scale.

evals, llm evaluation, ai quality, benchmarks and learning resources

Language Model Evaluation Harness

Language-model benchmark runner · EleutherAI · Intermediate to advanced

Reproducible few-shot and zero-shot evaluation of base or instruction-tuned language models on established academic benchmarks.

evals, llm evaluation, ai quality, benchmarks and learning resources

Lighteval

Multi-backend LLM evaluation toolkit · Hugging Face · Intermediate to advanced

Teams evaluating local, distributed, or API-served models across a large multilingual task catalog while retaining sample-level results.

evals, llm evaluation, ai quality, benchmarks and learning resources

SWE-bench

Software-engineering agent benchmark · SWE-bench team · Intermediate to advanced

Measuring whether coding agents can resolve real GitHub issues by producing repository patches that pass executable tests.

evals, llm evaluation, ai quality, benchmarks and learning resources