What this video shows
Agrawal describes a four-stage rollout for Duolingo’s iOS repository: local validation, scheduled backfill, pull-request lifecycle management and an engineer-triggered path for new code. Temporal coordinated work, S3 held shared state and cloud coding agents wrote tests and opened pull requests.
The speaker reports more than 700 merged test pull requests and 225,000 added lines of tests since March, with overall unit-test coverage rising from 11% to 24%. Those numbers come from Duolingo’s internal case study and do not show whether the same throughput or merge rate transfers to another repository.
Compare the workflow with Temporal’s durable execution model before choosing an orchestrator. Use GitHub’s protected-branch guidance to keep required checks and reviews outside the agent. Read Apple’s XCTest documentation for current test APIs and constraints.
What you will learn
- Start with files whose behavior and test style are already understood.
- Separate file selection, test writing, CI repair and final review so each stage has a clear contract.
- Block edits outside tests, mocks, fakes and stubs with deterministic checks.
- Record rejected pull requests and feed confirmed failure patterns back into repository rules.
How to apply this safely
- Choose one narrow test category and measure its current coverage, flake rate and review time.
- Run a small local batch and classify every rejected change before scheduling any work.
- Add deterministic path restrictions, required CI checks and a human approval gate.
- Limit concurrency and retries, then store workflow state so a worker restart does not duplicate work.
- Review merge rate, escaped defects, flaky tests, cost and reviewer time each week before expanding scope.
Important limitations
- The performance and coverage figures are speaker-reported results from one internal iOS repository, rather than an independent benchmark.
- Lines of test code and coverage percentages do not establish that tests catch meaningful regressions.
- Duolingo had repository-specific infrastructure, skills and reviewers that another team would need to build or replace.
- The talk covers generated unit tests and does not prove the same workflow is safe for production-code changes.
Sources to check
- 225,000 Lines of Tests, Written and Merged by Agents AI Engineer talk by Kush Agrawal, published November 11, 2026.
- Temporal documentation Official durable-execution documentation.
- GitHub protected branches Official required-review and status-check guidance.
- XCTest Apple’s test-framework documentation.
Continue learning on Learnetto
Build an AI agent test set
Define representative tasks and objective checks before scaling.
Review AI-generated code
Keep human review focused on behavior, risk and evidence.
Sandbox AI coding agents
Restrict files, commands, credentials and network access.