What this video shows
Cole Medin demonstrates the first production application built by his AI Dark Factory: an AI tutor that searches his YouTube videos and Dynamous community material, answers questions with retrieval-augmented generation (RAG), and links users back to cited source videos. The live stream tests the beta with real questions and files product issues while the system is running.
The more useful story is the delivery system behind the app. Medin reports 122 merged pull requests without personally writing or reviewing the code. Archon workflows move GitHub issues through triage, research, planning, implementation, validation, pull-request creation, merge, deployment, and browser checks. The stream also exposes the weak points: structured-output failures, stalled model calls, misleading status reports, incomplete evaluation coverage, and the need to observe costs and workflow state.
What you will learn
- The harness matters as much as the model. The repeatable issue-to-production workflow, permissions, tools, and checks determine whether autonomous coding remains controlled.
- Use separate sessions for planning, implementation, and validation. Medin argues that an independent validation agent is less likely to become a student grading its own homework.
- Combine agent reasoning with deterministic gates. Structured outputs, tests, browser checks, pull-request states, and deployment checks give later steps evidence they can verify.
- Code quality and operational reliability are different. A model may produce good code yet still break automation by ignoring a JSON contract, hanging, or returning an unreliable status.
- A working demo is not the same as a measured production system. The AI tutor could answer and cite sources, but its comprehensive RAG evaluation system was still planned rather than complete.
How to apply this safely
- Choose one low-risk repository and one narrow issue type before attempting broad autonomous delivery.
- Write the lifecycle as explicit states: intake, classification, research, plan, implementation, focused checks, independent validation, pull request, approval, deployment, and post-deploy verification.
- Run planning, implementation, and validation in separate agent sessions, passing artifacts forward instead of relying on one long conversation.
- Add deterministic checks for tests, linting, security boundaries, schema changes, deployment health, and browser behavior; make failures stop or reroute the workflow.
- Keep human approval before merge and production deployment until your own regression data shows that the workflow is safe for that repository and task class.
- Track model latency, hangs, retries, token or API cost, changed files, test results, pull-request status, and deployed revision so a silent workflow cannot be mistaken for a healthy one.
Important limitations
- Medin deliberately treats this as an experiment with no personal code or pull-request review before merge. That is evidence about his workflow, not a generally safe operating model; production permissions should match the consequences of failure.
- At the time of the stream, the tutor did not yet have a comprehensive RAG evaluation system. Successful example questions do not establish retrieval coverage, citation accuracy, or behavior across the whole knowledge base.
- The live run shows provider-specific stalls and structured-output failures. The named models, pricing, limits, and tool support are time-sensitive and should be rechecked before copying the stack.
- The reported 122 pull requests and results come from the creator during the demonstration; Learnetto has not independently audited the repository or benchmarked the Dark Factory.
Continue learning on Learnetto
Best coding agent courses
Learn the planning, tool execution, testing, and review loop behind coding agents.
Best AI agent evaluation courses
Build the traces, graders, regression datasets, and safety checks that the demo identifies as unfinished work.
Best AI engineering courses for developers
Go from a working LLM demo to a production system with observability, cost, latency, and reliability controls.
RAG learning path
Study retrieval, grounding, citations, and evaluation for an AI tutor like the one shown here.