/ LESSON 01 · 55m

Evals for Agent Builders

/ Curriculum notes

This lesson is available as written curriculum now. Use the notes below with the matching PDF workbook in the resources library.

Stop guessing whether your agent got better Workbook: /resources/eval-driven-development.pdf Codex route: /resources/eval-driven-development-codex-build-guide.pdf Claude Code route: /resources/eval-driven-development-claude-code-build-guide.pdf

/ Matching workbook

Open the PDF curriculum library

/ Choose your build route

Build this lesson inside Codex

Open the repo in Codex, let it inspect the files, then paste the prompt. Ask it to edit only the smallest set of files and verify before you deploy.

Before Codex

1. Open the project in Codex.
2. Confirm .env.local exists locally and is ignored by Git.
3. Open README.md and package.json so Codex can orient itself.
4. Do not paste private keys into the prompt.

Paste this prompt

Inspect this repo for the Evals for Agent Builders build.

Outcome:
Create an eval harness so prompt changes and model swaps do not silently break production.

Tools:
OpenAI, Claude, Supabase, PostHog, GitHub, TypeScript, Python, Codex

Explain the files a beginner needs to understand before editing:
README.md, package.json, src, public, scripts, .env.local, and any Supabase files.

Then implement the smallest safe version, list required env names, run the build or focused tests, fix failures, and summarize changed files.

/ Transcript

Evals for Agent Builders Outcome: Create an eval harness so prompt changes and model swaps do not silently break production. Tools: OpenAI, Claude, Supabase, PostHog, GitHub, TypeScript, Python, Codex Workbook: /resources/eval-driven-development.pdf Codex route PDF: /resources/eval-driven-development-codex-build-guide.pdf Claude Code route PDF: /resources/eval-driven-development-claude-code-build-guide.pdf Build assignment: Create ten eval cases for your first agent and run a before/after prompt comparison. Use the lesson tabs to choose Codex or Claude Code, then post the proof in Skool.

Tool Use and API Integration Patterns