Method · MoneyInclude examples of good and bad output — it most improves quality. Label examples with XML tags, otherwise the LLM treats the example as the thing to write and spits it back. Examples give the judge a concrete reference for your labels, dramatically improving consistency especially on edge cases; if output is too non-deterministic, an example grounds it in what you expect. A good actionable example report contains percentage revenue growth, a PE ratio, a con backed by a number, and a specific recommendation; a non-actionable one is correct but has no specific data or risks and just says investors should consider various factors.
Method, not a verified protocol.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Method · MoneyBuild golden datasets to measure whether your evaluator is actually doing its job. Give yourself real concrete criteria and stick to them when labeling — don't just say this was good, this was bad; be specific the same way you told the LLM to be specific. Test in both directions: have cases where a behavior should occur and cases where it shouldn't — if you only test whether it searches the web, you'll end up with an agent that always searches the web even when it doesn't need to.
Method, not a verified protocol.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Method · MoneySplit your labeled data into a development set (75%) and a test set (25%). Use the dev set to iterate on your eval — tweaking criteria and adjusting examples until the judge agrees with you. Hold out the remaining 25% to run the judge and see that it gets the scores you expect — that's how you know the judge is generalizing to new examples.
Method, not a verified protocol.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Method · MoneyLaurie Voss recommends using a coding agent (e.g., Claude Code) to pull down your failing traces, modify your code, and improve your application instead of hand-writing prompt fixes yourself. Claude pulls down all of the failing traces, reads all of the explanations, and turns the explanations into actions it takes on your code base. Nothing in the automated improvement is guessed — every change the coding agent proposes traces back to a specific failure explanation. For example, if the judge said the output lacks specific recommendations, the coding agent will rewrite the prompt so that it does; if the judge said the output presents risks without supporting evidence, it will rewrite the prompt to fix that too. This is data-driven prompt engineering, and it can now be automated.
Method, not a verified protocol.
AI Engineer · From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
Method · MoneyThe verifier first generates a rubric with N criteria of what success looks like, ranks the most relevant screenshots in the trajectory for each criterion, uses the top-K evidence group to determine whether each criterion was met, and outputs a scored rubric with partial credit (process score) plus an outcome Boolean with an explanation.
Method, not a verified protocol.
AI Engineer · Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase
MethodMaintain a high index of suspicion for differentiation syndrome in any patient starting a menin inhibitor: for revumenib, onset is roughly within one to two weeks; for ziftomenib it can occur later, up to one to two months, because of its longer half-life. Patients should come in for evaluation immediately if they develop fever, joint aches, bone aches, chest pain, trouble breathing, or rash. Workup includes checking blood counts for rising white count and differentiating cells, evaluating kidney function and joint/bone aches, obtaining a chest x-ray for pleural or pericardial effusions, and performing an echocardiogram if a pericardial effusion is found. At any suspicion, give high-dose dexamethasone 10 mg twice daily for three days regardless of symptom improvement — stopping steroids
Consequential creator advice requires independent verification before Today can promote it.
Method, not a verified protocol.
Research To Practice · Acute Myeloid Leukemia — An Interview with Dr Ghayas Issa on the Role of Menin Inhibitors
Method · MoneyShip an AI application (e.g., a docs page with an AI chatbot), let customers use it, and real production logs start coming in. Sample a subset of production logs — maybe 10 to 20 rows — to create a dataset. From that sampled dataset, write your task, write your scoring system, and run evals; tweak your dataset, prompt, or score and run multiple different evals. After running a couple of evals and seeing which does best, you learn something about your system, make the code change, deploy it back into your application, and the whole flow repeats because those changes show up in your production logs.
Method, not a verified protocol.
AI Engineer · Agentic Search vs Vector Search for Coding Agents: We Ran the Eval — Braintrust
Method · MoneyOpen with the 'heard the name tossed around' opener — 'Hey [name], we work with a few other [portfolio company type] companies. It's [name] from [company]. Have you heard our name tossed around?' — mentioning mutual connections or shared context before pitching. Then lead with the problem before the solution (a 'problem proposition' or 'reverse pitch'). For objections, use the 'Mr. Miyagi method': agree with the objection and sell a test drive before fully addressing it.
endorsed
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Method · MoneyStart with a 15-minute recruiter screen to filter out candidates who cannot compose a sentence or show basic professionalism, protecting the team's time. In the hiring manager interview, assess whether the candidate is 'smart' and 'cares' by asking why they want to get into sales and what they did to prepare. Then run a mock cold call twice: the first call will suck and that doesn't matter — evaluate whether the candidate listened to prep material and tried to apply it, and whether the gears are turning. Give no more than two points of feedback, coach them to lead with a problem statement instead of a pitch, then run a second call to see if they adapt. If they don't change anything on the second call, that's a massive red flag. After the mock calls you should know if you have a 'triple A'
endorsed
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Method · MoneyRun an SDR team on five weekly meetings: Monday morning meeting (energy + activity focus), Tuesday teardown (cold call/email/account review), Wednesday one-on-ones (individual coaching), Thursday teardown, and Friday team dials (energy + activity). Bookend the week with energy-focused meetings on Monday and Friday to keep the team dialing.
anecdotal
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #606 - How To Build An Elite Outbound Sales Team (Get Reps Dialing Fast)
Method · MoneyA concrete multi-step method: (1) go to the company's leadership page and manually add CEO, CFO, SVPs; (2) use LinkedIn and Sales Navigator to find additional CXO/SVP/VP-level people; (3) guess reporting lines (e.g., Scott reports into Jordan, COO Valentina reports into Jordan); (4) add people like Fatima and Nadia based on best guess; (5) make up an excuse for a one-on-one with the champion and walk them through the guessed chart so they correct it. Laura prefers doing this in a tool called 'Mirror' and has built charts down to six layers in some organizations.
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #608 - How Mapping the "Shadow" Org Chart Got Me to 305% of Quota
Method · MoneyA multi-step method for building multi-threaded executive relationships before a deal closes: map the customer's org chart and power base in a Miro board, seek to be corrected to learn the real org chart, get introductions from the champion to key people by framing why access is in the champion's interest, then play matchmaker by pairing peer levels (CEO to CEO, engineers to engineers, C-suite to C-suite) using the EA, a ghost-written message, and a prep doc.
The hosts present this as an endorsed playbook for enterprise deals; no outcome data is cited.
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #608 - How Mapping the "Shadow" Org Chart Got Me to 305% of Quota
Method · MoneyRun AM discovery in three steps: (1) recap results already achieved from the last project — start with the trust factor ('this is what you asked us to do and this is the result we got') rather than a hypothesis, unlike AE-style discovery; (2) run discovery for the next outcome; (3) come back with a next step to solve it. At the end, tell the customer you'll return in a week with a plan built with the CSM and product expert showing how they can reach the outcome using the product.
Commercial context: The creator presents this as their own offering and directs viewers to get it.
Method, not a verified protocol.
30 Minutes to President's Club | No-Nonsense Sales · #604 - Upsell & Expansion Masterclass: 7 Lever to Grow Any Account
Method · MoneyA coherent, ordered method from the talk: (1) Give the agent Markdown files — docs checked out locally, plus major website pages (changelog, pricing, features) converted to Markdown — so it understands everything about the business and stays perfectly in sync with shipped features; (2) provide all context as a large Claude MD instructions file (~30,000 lines) containing guardrails and operating advice, and ship an update to it whenever the agent gives a wrong answer or has a bad experience, so every future conversation improves; (3) show the agent the URL of its sources so it can cite them to customers for verification; (4) use a dedicated embeddings partner (Voyage) so the agent gets the most relevant documents up front and answers faster; (5) containerize the agent as a Docker file and (
Method, not a verified protocol.
AI Engineer · We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI
Method · BodyStart with body weight single leg squat progression. Progress to overhead and goblet single leg squat variations. Progress to the Zurcher single leg squat variation. Progress to advanced VBT training in the single leg squat, pushing it as heavy as possible.
Method, not a verified protocol.
Garage Strength · The "Single" Best Lift Every Athlete Should Do!
Recommendation · MoneyLincoln Murr recommends giving an agent a wallet or financial account so it can pay for different APIs on a one-off basis, instead of pre-funding accounts and managing API keys.
AI Engineer · Most of Your Docs Readers Are Now AI Agents - Hahnbee Lee, Mintlify & Lincoln Murr, Coinbase
Recommendation · MoneyJohn Stecher says that if models will be good enough and compute sufficient in six months, delay building the application today and address it with the new model — it will cost less, provided the need for the application isn't pressing. He also cautions that you don't want to wait too long to build something, because you can always be waiting for the new model or new technology.
AI Engineer · Why Waiting for the Next AI Model Can Pay Off - John Stecher, Blackstone & Scott Worland, Norm AI
Recommendation · MoneyJohn Stecher recommends using agents to help understand existing human workflows, then writing code to optimize for straight-through processing and injecting agent intelligence only where humans previously made decisions. He also advises stepping back to identify work agents can do better, to offload intellectual cycles from investment, finance, and engineering professionals so they can do higher-value work.
AI Engineer · Why Waiting for the Next AI Model Can Pay Off - John Stecher, Blackstone & Scott Worland, Norm AI
Recommendation · MoneyContinuously test the workflow at every part of the workflow, designing the test to that specific part and measuring success in that element. For a 100-step workflow, each step must be evaluated for what goes into it, what comes out of it, and the expected outcome. Testing only the final output misses intermediate failures, like an image edit that changed the background but added unwanted handles. Assuming input and output to be the outcome of anything in the workflow leads to untested failure points.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyCharlie Holtz advises creating 'slot-free zones' — parts of the code base or app that require really strict human review. He warns that if you're not careful about slot-free zones, your code base can get in a really tricky spot, and notes Conductor has had to rewrite their whole app a couple of times because of this. As a concrete example, any change to the migrations file in Conductor's CI requires a human to review it. He also notes this careful-in-places approach is unusual: most people assume top builders are pure token maxers ripping through 30,000-line PRs.
AI Engineer · Orchestras, Not Factories: How the Fastest Builders Work — Charlie Holtz, Conductor