37,617 results · Most relevant
When agents can generate thousands of lines of implementation detail, humans need a compressed review surface. A good diagram lets us see the shape of the system in a way code often obscures. Use diagrams not as decorative documentation after the fact, but as a way of making systems legible.
Users understanding privacy, their data, and how it's used is way more important than it always has been, as is users understanding the norms of your product. Before, you could get away with unpredictable behavior because there was a settings menu or a pop-up that could appear. When the agent is doing things semi-autonomously, you really need to make sure that at any given point a user can predict what is probably going to happen with a feature or primitive you're introducing.
endorsed
The speaker argues that AI content all sounds the same and will feel like spam without human judgment, so the voice, taste, and judgment have to come from him. He also explains that automatically sending AI-generated emails leads to broken drafts reaching recipients — which is why the pipeline does not automatically send emails, and drafts are reviewed manually before sending.
Design AI agent products so the human reviews and approves the agent's plan before execution: the user states intent, the agent returns a plan, the user edits it, and only then does the agent execute. This keeps the vendor responsible for outcomes while making the right choices obvious to the user.
Charlie Guo's advice for GOAL: be really specific with the criteria you give the model for knowing when it's done, and make that completion criteria as verifiable as possible. When given a GOAL, the agent keeps evaluating itself against the success criteria, checking at each turn whether the task actually completed and whether to keep going. He compares GOAL to a genie: when it works well it is insanely magical, but when it doesn't it can be a monkey's paw situation where it technically did what you asked but not in the way you expected. GOAL suits very large projects — code migrations, large refactors, retry loops, experiments, games, and one-shotting apps with detailed specs — and he cites Thomas's slash GOAL that ran 40 hours reimplementing Doom in native Swift.
endorsed
Laurie Voss calls reading your traces before you write evals the most important practice in the entire workshop: before writing a single evaluator, start with your data and read a dozen or more traces end-to-end. When reading traces, focus on the ones where something went wrong: what was the input, what was the output, what specifically is broken. Reading raw inputs and outputs sounds old-fashioned but is one of the highest value activities in agent development. A trace tells you whether the agent made a genuine mistake or whether your graders rejected a valid solution. Anthropic invested in tooling specifically for viewing eval transcripts, and their team spends a lot of time just looking at eval transcripts every day.
The speaker advised grading whether the output had the right information and whether the final state matched what you wanted, regardless of how the agent found the data; checking a specific sequence of tool calls is too rigid in practice, and grading the path punishes creativity because agents find valid approaches the eval designer didn't anticipate.
Writing the eval first gives a clear, measurable definition of done; then build the feature until the eval passes. Some teams write evals at the very start, others add them at scale — either way treat evals as core.
When an agent's system changes (model, prompt, tools) and behavior might drift, the solution is evals — specifically regression evals already in place before the change. Running them confirms the system doesn't change from previously good behavior, so you rely on evals rather than manual inspection.
Eval failure explanations can be fed back into a coding agent, which can then make the software better automatically and close the software development loop without manual intervention. Once you have defined what good looks like, your agent can climb the hill by itself.
Rob Walling argues that founders should validate an idea before committing to build, launch, and market it. Validation means doing research, putting thought in, having conversations, or testing the assumption that what you want to build is valuable. He frames this as a way to avoid spending months or years building something nobody wants, and notes that serial entrepreneurs do more validation before selecting an idea than first-time founders who flail. He also cautions that validation can never reach 100% certainty — you only get that by building something and selling it.
Reliability is the primary value driver for AI tools. Each additional 'nine' of reliability (e.g., 99% to 99.9%) unlocks new applications. Developers should prioritize background reliability over flashy demos.
A calm, unemotional energy that holds the line on agreements builds respect and makes deals work better, unlike a 'no problem' calm that lets things slide and leads to being walked over.
Karri Saarinen recommends having a shared place for learning about customer context, product thinking, and anything else the team is doing — a shared space where context lives around the customers, the product, and the work happening around the product. He stresses this is important not just for people but also for AI agents, and you can ask agents questions like 'what do we have about this?' or 'teach me something about the customers — what are they saying, what are they doing, and how is the team responding?'
As part of the product engineering process, create a Hilltop document — a PRD that defines the purpose of the project and incorporates what customers are talking about and where the need for the unit of work is seen. It includes competitive analysis (whether there are similar products in the market to draw inspiration from), brings early design screens into the document, and outlines the major milestones. By consolidating and canonicalizing project information into a Hilltop document, you can give that resource to an agent and the agent can break it into units of work. TARS can see through Linear tickets that a Hilltop document has been reviewed and approved, and by virtue of approval, start work on the project automatically — so you don't need a human shepherding the agent through every
The source argues that while AI will be a phenomenal tool in the physical world, AI cannot turn a wrench yet, and physical-world AI strategies must center on software augmenting human-operated processes. Applying a shiny AI product as a silver bullet over a physical process you don't deeply understand is described as an anti-pattern; success requires deep understanding of the physical process. A concrete example is given: manufacturing engineers can now use AI tools like Claude to reprogram and reconfigure manufacturing lines on the fly, retooling a line in about 20 minutes instead of the three months and hundreds of thousands of dollars a systems integrator would take. Technicians' physical-process intuition plus these new software tools changes the economics of low-volume, high-mix, hard
Jimmy Carr recommends testing new material at small club shows — like the mothership's 100-110 seat room — to generate premises that can become gems in your act; he says three or four bits of his current act came from premises pulled out of the barrel.
Because nobody can hand-audit 300 skills for drift, Orobator recommends a scheduled pass — an agent 'bedtime' — where the agent reads its own skills, finds stale ones and contradictions, and opens drafts for review. The system sleeps, cleans house, and wakes up sharper, improving over time.
For repetitive robot tasks, do not keep the LLM in the loop; write code that can run repeatedly very fast instead. If you've done a similar robot task before, call a compiled skill that handles the task faster and manages edge cases better. Automate the repetitive parts of a robot task using code since they become deterministic after doing the task a few times.
hedged
Put a VLM inside the loop to add points of variation to a deterministic coding graph so it can generalize. Build variation points into the code-as-policy graph for object detection and failure handling so the system can respond flexibly.
Choose sources → to get a library of your own and a daily brief.