Creator perspective · MoneyApplying the why-why-why method to the facial recognition request reveals the underlying problem: check-in is slow. The core job of the app is getting people into the room and knowing who came in, and the job statement has nothing to do with facial recognition — understanding it informs what to build.
AI Engineer · Build the Right Thing: Product Engineering (Part 2) — Kent C. Dodds, EpicProduct.engineer
Report · MoneyLaurie Voss explains that online evals take the evaluators you wrote (ticker check, faithfulness, actionability) and run them automatically on incoming production traces — the same evals running on new data.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Creator perspective · MoneyThe source describes a case where a voice agent processed a refund for order 40 instead of order 14, and the transcript looked fine but the call wasn't, making it the canonical example of transcript-vs-audio failure.
AI Engineer · The Transcript Looked Fine. The Call Wasn't. — Debugging Voice Agents, Arize
Report · MoneyImplementing observability in an OpenAI Agents SDK application is as simple as turning it on, because the SDK is already instrumented with OpenInference. The agent was told to send the traces it was already generating to the Arize endpoint with the API key, and after a restart and a query, traces were flowing into the Arize project with no code written.
Commercial context: The creator presents this as their own offering and directs viewers to get it.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI
Report · MoneyEven though evals compress thousands of traces into eval scores, at 2026 scale you get so many evals running that you cannot stay on top of them as a human by yourself. The proposed solution is to have an agent read eval failures en masse to detect signals and patterns — for example, identifying that 100 of 1,000 failed requests are the same failure — and fix that pattern.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI
Report · MoneyWhen asked for bomb-building instructions, the demo agent said it couldn't help but then recommended bomb-related toys — a harmful-query failure. The correct behavior is to ignore such queries entirely rather than trying to be helpful, and a basic safety eval ensuring the agent never gives bomb-building instructions should have existed from the beginning.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI
Report · MoneyThe speaker asked the agent to use the skills he gave it to pull down his traces from Arize AX directly; the agent has the API key and the endpoint. A default off-the-shelf coding agent would not know how to pull and analyze traces — the installed skills are what give it that knowledge.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI
PrincipleDr Ghayas Issa advises that the key factor for improving durability of response — hopefully cure — in relapsed/refractory disease treated with menin inhibitors is consolidation with a stem cell transplant.
Consequential creator advice requires independent verification before Today can promote it.
Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors
Report · MoneyIn the live demo, the scenario tested was: 'I purchased headphones 20 days ago, I want to return them. Can I?' The correct answer was that you cannot return them (return window is 14 days), and the test expected the answer to contain 'cannot'. The speaker wrote a unit test for a TypeScript function add(num1, num2) that should return the sum, and ran it with npx vitest. The test iterated over each candidate answer with answers.forEach and expected each answer to contain 'cannot'. The scenario answer 'yes, you can return them for free' failed the test, while 'no, you cannot return them' passed. The demo answers were generated by an LLM, which is why the test was non-deterministic. Fuzzy matching on a keyword like 'cannot' is non-deterministic and breaks: an LLM-generated answer like 'you
AI Engineer · Evals in AI: A Deep Dive — Tejas Kumar, IBM
Report · MoneyThe full eval pipeline journey: start with a unit test, make it fuzzy, add a judge, train the judge, put it in production, gate on agreement, keep it living by skimming data and adding it to the data set, and even discover policy automatically. The speaker ran a naive job five times with a pairwise prompt asking which reply is helpful, and the results were a mixed bag with all wrong answers. When the policy is included in the prompt and the eval is run five times, the results are suddenly correct. The speaker built a more aesthetic demo of the eval system as a running application (npm run dev on port 5173).
AI Engineer · Evals in AI: A Deep Dive — Tejas Kumar, IBM
Report · MoneyEven after giving the model the right context, a low-quality model may still not behave — at that point you must consider a better model. A very low-quality model produces extra problems like case-mixed responses, signaling you're under the floor of quality. Once you've maxed out context and policy work and the model still misbehaves, that's the right time to switch to a higher-quality model — but you don't have to go crazy. Switch to GPT-4.0 Mini as a higher-quality model upgrade for eval use cases, since it is practically free and token usage is low. GPT-4.0 Mini produces consistent casing between pass and fail responses, making it a significant upgrade at a good price. When you bump the model to a higher-quality one without changing anything else, the eval agreement improves
AI Engineer · Evals in AI: A Deep Dive — Tejas Kumar, IBM
Report · MoneyHarald Kirschner reports that GitHub issues are a massive signal for holding quality when you're on GitHub. They get sometimes even better issues now because of AI and more people with different language backgrounds being able to file very good issues. Previously, engineers actually triaged issues manually. Now AI does the filtering to get out spam, enriching the data, and then translating, in the end assigning area owners for all the issues to raise the signal-to-noise quality. Add a human layer to AI issue triage — once you are in the issue, because the agent will make mistakes and we need that feedback loop.
AI Engineer · How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner
Report · MoneyHarald Kirschner reports that the VS Code team started tracking code survival metrics early on — the percent of agent-written code that actually gets committed. GPT 4.1 started at 55% code survival, and Claude Opus 4.6 is now at 86%. The increase shows developer trust and confidence in shipping AI code faster and more AI code generated.
AI Engineer · How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner
Report · MoneyHarald Kirschner reports that code review is the next big investment. When they enabled the automatic GitHub Copilot code review, they weren't convinced yet, but over the next months and weeks it massively improved. Now it's a mandatory review every time somebody opens a PR. After the AI review is done, humans will not even look at a PR until all the comments are resolved and addressed. You can dial in the effort a code review does depending on how much risk you have in the repo — low, medium, high effort for code reviews — to dial in the cost benefit.
AI Engineer · How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner
Report · MoneyThe initial architecture routes messages from an iPhone or MacBook through WhatsApp to NanoClaw on the Raspberry Pi, requiring a credential registration on NanoClaw so it can pull messages from WhatsApp in the cloud. No LLM inference happens on the Pi — Jeremy uses the cloud for all inference, describing it as 'a big brain over a wire to the cloud' — and he also has a Neo4j database in the cloud alongside the inference.
AI Engineer · I Built a Personal AI Agent on a Raspberry Pi — Jeremy Adams, Neo4j
Creator perspective · MoneyThe final system: a request comes in, a serverless authentication function authenticates the browser, the authenticated browser is provided to the agent runtime, the agent navigates the portal using the skill, calls a deterministic download function, and provides the downloaded file to a verify tool so it can know authoritatively whether its trajectory succeeded or failed. This architecture takes steps out of the model's responsibility and, Meegan argues, can overcome the browser agent conundrum.
: none
AI Engineer · Why 99% Accurate Browser Agents Still Fail — Derek Meegan, Browserbase
Report · MoneyCurrent protocols like MCP are stateless, making sticky sessions difficult. A2A is client-server based, requiring complex dual-role setups for bidirectional communication, and lacks built-in discovery mechanisms, forcing developers to handle planning and infrastructure manually.
AI Engineer · Your Agents Are in Solitary Confinement: Why MCP & A2A Aren't Enough — Vlad Luzin, Band
ReportCurrent AI coding agents (e.g., Cloud Code, Codex) primarily accelerate the production of existing software types rather than expanding software capabilities. They automate minor tasks like small pull requests (~10 lines) but do not create fundamentally new software functions.
high
The a16z Show · AI Can Write Code. Why Isn’t Software Better?
ReportThe host describes the second issue of i-D featuring a baby on the cover, representing Gen Beta, and notes the baby was photographed winking. In lieu of a profile, the host asked important artists, politicians, and figures to give advice to someone born in 2025.
Figma · 8 Designs That Inspired i-D Magazine's Editor in Chief | Source Material
ReportThe host describes the first issue of i-D magazine under their editorship, which featured a nationwide casting call to find an unknown cover star. They found Enza, a senior in high school in suburban Cleveland, Ohio, who was an aspiring model and LARPer. The host and photographer Aiden Zamiri spent three days visiting her, hanging out and seeing her world, then shot her in her environment: her room, her living room, and at her drama club.
Figma · 8 Designs That Inspired i-D Magazine's Editor in Chief | Source Material