Recommendation · MoneyKent C. Dodds recommends validating that the problem exists and is interesting before building a solution. In the pre-launch era with no users and no metrics, the Mom Test comes in handy. With AI you can prototype things really quickly to validate problems. After validating, build a simple solution and see if people resonate with it; watch people use your software, and if you can't get anybody to try it out, that's a signal. You want to find something important enough that somebody wants to try it out.
AI Engineer · Build the Right Thing: Product Engineering (Part 2) — Kent C. Dodds, EpicProduct.engineer
Recommendation · MoneySet online evals to run continuously against new incoming traces in production. If evals are expensive or you have high traffic, set the sampling rate to evaluate only a percentage of traffic — e.g., 1% — which still gives a whole bunch of eval data without breaking your budget.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Recommendation · MoneyBuilding one big mega evaluator that checks everything at once is a nightmare to calibrate — when a 'God evaluator' says fail, you can't tell which of the five dimensions failed. Split evaluations into one evaluator per dimension; the combination of multiple focused evals gives a richer picture than any single eval ever could.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Method · MoneyTo add observability to an agent app, ask the coding agent to add observability with Arize AX to the application (the speaker asked: 'Can you add observability with Arize AX to this application? You've got secrets in .env.local already.'); to get observability out of an application you only need to turn on tracing — tell the app you're using Arize, provide your API key and endpoint, and send the traces it already generates to your endpoint; implementing observability in an OpenAI Agents SDK app is simpler than expected because the SDK is already instrumented with OpenInference; after adding observability, restart the app and run a query to verify traces are flowing, and to verify tracing is configured, check your .env.local file for the tracing project name and look for that project in the
Commercial context: The creator presents this as their own offering and directs viewers to get it.
Method, not a verified protocol.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI
Method · MoneyTo have the agent analyze your traces, ask it to pull your traces and look for things that are going wrong ('Can you pull my traces and look for things that are going wrong?'); to fix a specific problem found in traces, ask the agent about that problem in particular and whether it can fix it ('What about that problem in particular? Can you fix that problem in particular?'); after the agent fixes the problem, verify the fix by re-running the failing query and checking the traces show correct input, output, and response; throughout the demo the speaker never touched the code — all he did was say 'look for problems' and 'fix that problem in particular'.
Method, not a verified protocol.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AI
Recommendation · MoneyAnother way to verify verifier quality is to train on data filtered by it: hold the number of training examples constant (3K or 9K trajectories) and filter them by whether the verifier judged them as passed. When doing SFT, train only on successful trajectories (where the verifier marked them true).
AI Engineer · Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase
Recommendation · MoneyCharles Frye recommends speculative decoding as a key technique: take a separate model (the speculator) to guess the next couple of tokens, pass them through the target model, and grade them in parallel — turning decodes into tiny prefills. With rejection sampling applied, the target model's output is guaranteed to be the same as sequential decoding, up to numerics. As you improve the quality of the speculator and the number of tokens it can guess in a row, you get a basically linear speedup in decode throughput.
Also discussed by Dan (speaker, name from Q&A) on Y Combinator, Y Combinator (unclassified source — not counted until classified). Not counted as independent sources.
AI Engineer · What Is an Inference Engine, Anyway? — Charles Frye, Modal
Recommendation · MoneyCharles Frye recommends CUDA graph capture as a key technique for reducing CPU-side launch overhead: track all kernels in the model forward path, turn them into one big DAG of operations, and launch them from a single graph instead of launching each kernel individually. This turns per-kernel CPU launch work into a single CPU-side decision of which graph to launch, relaxing constraints in the model forward pass.
AI Engineer · What Is an Inference Engine, Anyway? — Charles Frye, Modal
Recommendation · MoneyBlaine Billingsley recommends staying vigilant about keeping your system small — the bigger things get, the more likely you won't be able to guarantee their success. Make sure your product solves one thing absolutely perfectly at small scale before cautiously expanding, verifying it doesn't fail at each step up.
Design Better · Blaine Billingsley: OpenAI designer on the collaboration problem AI hasn't solved yet
MethodDr Ghayas Issa advises that at any suspicion of differentiation syndrome, give high-dose steroids: dexamethasone 10 milligrams twice a day for three days, regardless of whether symptoms improve, because the goal is to prevent the syndrome from recurring. A dose of steroids can be diagnostic — patients often feel 100% better with the first dose, confirming you're on the right track. Because differentiation syndrome can be deadly, it is better to err on the side of giving steroids than to wait to confirm the diagnosis and risk it being too late to correct.
Consequential creator advice requires independent verification before Today can promote it.
Method, not a verified protocol.
Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors
MethodDr Ghayas Issa outlines the evaluation for suspected differentiation syndrome: check the blood count for an increase in white count (sometimes associated with differentiation); look at whether leukemia cells are turning into normal cells; a chest x-ray may be done because pleural or pericardial effusions can occur; an echo may be performed if there is a pericardial effusion; and kidney function should be evaluated.
Consequential creator advice requires independent verification before Today can promote it.
Method, not a verified protocol.
Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors
MethodDr Ghayas Issa describes the workup for suspected differentiation syndrome: check blood counts for an increase in white count (sometimes associated with differentiation) and look for leukemia cells turning into normal cells; obtain a chest x-ray because pleural or pericardial effusions can occur; perform an echocardiogram if there is a pericardial effusion; and check kidney function.
Consequential creator advice requires independent verification before Today can promote it.
Method, not a verified protocol.
Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors
Method · MoneyArmand describes the call structure he taught SDRs: an opener, then when the prospect says no, leading into a 'problem proposition' that leads with the problem before the solution (illustrated by his Carta example about CFOs finding it ridiculous to pay lawyers to issue stock options). For the spreadsheet objection, he used the 'Mr. Miyagi method' of agreeing with the prospect and selling a test drive. He reports that these three things got SDRs good on the phones, moving cost per appointment from $50 to $200 and meetings per SDR from four to eight.
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Method · MoneyArmand describes five meetings that can run 90% of SDR teams: a Monday morning meeting (MMM) that injects energy and focuses on activity, Teardown Tuesdays and Thursdays (30-minute sessions reviewing three cold calls, two cold emails, or two to three accounts, giving coaching without adding much calendar time), Wednesday one-on-ones for individual coaching, metrics review, and role play, and Friday team dials that end the week with energy and focus on activity plus culture. The week is bookended by energy-focused meetings on Mondays and Fridays with skill-focused sessions on Tuesdays and Thursdays.
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Recommendation · MoneyBefore addressing the team publicly, conduct 16 hours of back-to-back one-on-ones with every team member, using a 'between you, me, and the wall' conversation to privately share the turnaround plan. Winning over individuals first means peer pressure helps win over the rest of the group.
anecdotal
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Method · MoneyArmand describes a revamped SDR interview process. A recruiter screen cuts about 50% of the applicant pool. The second step is a mock cold call, where the first call is expected to be bad; the interviewer looks for whether the candidate's gears are turning and how much they researched, gives no more than two points of specific feedback (e.g., leading with a problem proposition instead of a pitch), and treats no change on the second call as a massive red flag. After the mock cold call the interviewer should know if they have a 'triple A' hire. The final stage is a triple onsite of three back-to-back interviews: a mock cold email (written live or drafted beforehand, testing whether the candidate applied prep and used their brain), a rep filtering test (narrowing 40,000 companies to 25, with
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Recommendation · MoneyA prospecting turnaround can often be achieved not by increasing activity or improving phone skills, but by removing waste in the prospect list. Categorize personas into A/B/C tiers and replace all C-tier prospects with more A-tier prospects without increasing overall activity. Don't reach out to personas that are a poor fit for your product (e.g., selling stock option management software to a Chief HR Officer).
endorsed
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Recommendation · MoneyPienaar recommends generating many different theories simultaneously to give the agent breadth and diversity of ideas, calling this Cleric's primary technique. The rationale: agents are excellent at relative ranking but terrible at absolute scoring of their own theories, so multiple candidate ideas beat asking for one scored answer.
AI Engineer · Your AI Agent Is Confidently Wrong About Production — Willem Pienaar, Cleric
Recommendation · MoneyPienaar recommends computing a baseline for production — extremely important in large environments — because most prod environments are in permanent disarray and chaos: engineers know which error logs to ignore and which metrics are unused, but an agent doesn't. Concretely, cluster logs and give the agent statistics on which logs recur daily and are common, not just raw logs; the same approach applies to metrics, object state, and many other things.
AI Engineer · Your AI Agent Is Confidently Wrong About Production — Willem Pienaar, Cleric
Recommendation · MoneyPienaar recommends using a topology or build dependency graph of the production environment. When the agent claims a causal connection between services, output all connections that actually exist and have another agent critique the claim — an effective way to introduce doubt and push the agent to go further if the claim doesn't map to a real connection.
AI Engineer · Your AI Agent Is Confidently Wrong About Production — Willem Pienaar, Cleric