Creator perspective · MoneyNew models are released every couple of weeks, making evals more important every month. Without evals you have no way of knowing which things a new model is better or worse at before switching. With evals, you run the suite, compare the numbers, and know whether you can upgrade your model. The workshop starts with Claude 4.5 and upgrades to Claude Haiku 4.5 (released about a week prior) to demonstrate model comparison via evals.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Creator perspectiveFailing to restrict agent network access via environment configuration leaves API keys exposed to potential theft; restricting it keeps the key 'hidden away from any potential theft'.
AI Engineer · Why One AI Agent Isn’t Enough (And How to Run 1,000) - CJ Avilla & Harrison Stall, Anthropic
Creator perspectivePutting an API key into a markdown file given to Claude is a security problem — 'that's definitely bad. Do not do that.'
AI Engineer · Why One AI Agent Isn’t Enough (And How to Run 1,000) - CJ Avilla & Harrison Stall, Anthropic
Creator perspectiveStoring API keys as plain-text files on the sandbox disk — even hidden files or secret directories like 'myapikeys.md' — always carries a risk of leaking, whether through prompt injection or other means; Anthropic built abstractions so credentials never have to touch the sandbox.
AI Engineer · Why One AI Agent Isn’t Enough (And How to Run 1,000) - CJ Avilla & Harrison Stall, Anthropic
Creator perspectiveGiving an agent unrestricted internet access — with WebFetch/WebSearch enabled by default — without explicit permission risks the agent reaching unintended domains; you don't want any internet access without explicitly allowing it.
AI Engineer · Why One AI Agent Isn’t Enough (And How to Run 1,000) - CJ Avilla & Harrison Stall, Anthropic
Creator perspective · MindBrooks walks through an exercise of eliminating idols one by one — power, pleasure, money — to reveal what remains: honor. He notes that if asked to name your idol at the start, you might not have arrived at the same answer; the elimination process itself reveals it. Aquinas suggested that once the other idols are eliminated, what you've found is what can make you stumble and distract you from love in your life.
presented as Brooks's endorsed method
Andrew Huberman · The Art of True Happiness | Dr. Arthur Brooks
Creator perspective · MindBrooks identifies the four pillars of a satisfying life as faith, family, friends, and work — to love and be loved means spending time on faith (love of the divine), family (people biologically or by adoption related to you), friends (the people you actually choose), and serving the world through the way you earn your daily bread. Only when you know your four pillars can you pivot to what your soul actually hungers for, and that's where real satisfaction comes.
presented as Brooks's endorsed framework
Andrew Huberman · The Art of True Happiness | Dr. Arthur Brooks
ReportFigma designs can be taken into Figma Make to convert them into real website-driven code; when someone questions whether a Figma design can be turned into code, the answer is yes; MCP (Model Context Protocol) can be used to take Figma code over to Codex, Claude, or another external AI agent for building, and for complex animations MCP with AI or another method can hand off the animation.
Figma · Figma for Edu: Prototyping with animations in Figma
Creator perspective · MoneyWhen using Claude or Codex on the same task, people commonly run multiple stateful sessions (one plans, another reviews; one writes code, another reviews) — effectively a multi-agent system where the human acts as a Cisco router moving packets between the two stateful agents.
AI Engineer · Why Your AI Agents Can't Talk to Each Other (Yet) — Vlad Luzin, BAND
Principle · MindLiemandt argues that a chatbot alone does not produce the learning improvements — generating a personalized lesson for every kid does. Lessons must respect prerequisites (don't give a chemistry lesson to someone who doesn't know algebra), verify student motivation and attention before judging effectiveness since disengaged students distort results, and use spaced repetition with later recall testing to verify the lesson actually transferred to long-term memory.
Andrew Huberman · How to Accelerate Learning & Improve Education | Joe Liemandt
Principle · MindLiemandt lays out a dependency chain: chemistry is basically algebra with cool word problems, so an algebra test predicts chemistry performance; algebra is fast manipulation of fractions; fractions come down to memorized multiplication and division tables. Math more than anything — and reading with vocabulary — is layers built on foundations, and the time to learn algebra is not in the middle of a chemistry problem.
the guest's endorsed heuristics
Andrew Huberman · How to Accelerate Learning & Improve Education | Joe Liemandt
Principle · MindGifted kids have the most working memory slots but do everything in their head and never write down their work; every gifted kid eventually hits a wall because some math can't be done in your head, and they learned bad habits instead of good ones. The remedy: write down your work on multi-step problems.
Andrew Huberman · How to Accelerate Learning & Improve Education | Joe Liemandt
Principle · MoneyKent argues that with AI agents, every engineer is now a team lead over however many agents they can run at once. Just as a team lead's traditional job was to make the system efficient for engineers — good testing, good architecture, obvious Lego blocks — it is now the engineer's job to make sure the playground is nice so agents don't make a ton of mistakes. Engineers should take great care in the system they create for their agents to ensure success.
AI Engineer · Build the Right Thing: Product Engineering (Part 1) — Kent C. Dodds, EpicProduct.engineer
Report · MoneyThe Instagram chatbot security failure could have been prevented at both the harness layer (just-in-time) and the eval layer (ahead-of-time); ideally evals would have caught it before deployment. An eval can verify that a chatbot confirms the user is who they say they are, regardless of how much they ask to change or where they log in from.
AI Engineer · Evals in AI: A Deep Dive — Tejas Kumar, IBM
Recommendation · MoneyJosh Gruenstein describes Tutor's founding thesis: deploy robots into real-world work first, then use that work to build labeled datasets, rather than trying to build intelligence from no data.
The a16z Show · The Robots Already Working in American Factories | Tutor Intelligence
Recommendation · MoneyJosh Gruenstein argues robotics companies need to build a full-stack platform spanning model, hardware, software, supply chain, customers, and product, because success at any single layer requires the combination of all three core layers.
hedged
The a16z Show · The Robots Already Working in American Factories | Tutor Intelligence
Recommendation · MoneyJosh Gruenstein advises starting robotics companies from what the robot is supposed to do at the end of the process, rather than from model or data insights, noting that few founders think about the end use case.
The a16z Show · The Robots Already Working in American Factories | Tutor Intelligence
Recommendation · MoneyNick Nisi recommends that when something breaks after a merge, the rule is to audit the whole pipeline, review what happened, fix the team review skill or the automated reviewer that missed it, and then use AI to fix the code going forward. The audit question to ask when a PR fails a check is: 'can we fix the team review skill to catch it, or fix one of the automated code reviewers to catch it the next time?'
hedged
AI Engineer · I Made a Fruit Fly Review My Code - Nick Nisi, WorkOS
Recommendation · MoneyIf you're a non-technical founder, make sure you have access to the source code and the servers running your infrastructure, so you're not creating a key person risk with essential business systems.
MicroConf · What to Outsource (and What to Keep): A Bootstrapped Founder Framework
Recommendation · SocialCoach Kyle endorses preparing flavorful answers for the predictable questions that come up time and again — work, what you're up to, where you're from — instead of flat, boring responses. Examples include framing your job as the story of what you initially wanted to do and the experiences that led you there, mentioning the luxury of taking Fridays off, or that friends recommended checking out the city center. A flat answer signals you have nothing fun going on in your life.
Coach Kyle · https://www.youtube.com/watch?v=AFD-p4vgrwk