Recommendation · MoneyFor each eval task, Ramp used 10 to 20 criteria to define success. When building the benchmark, they created a set of task categories and defined what success looks like for each. In the reconciliation example, they identified a few things that define success — which, as an MLE, he would call labels.
AI Engineer · How Ramp Tests Its AI Bookkeeping Agent Before Customers Do - Ryan Stevens, Ramp
Recommendation · MoneyInstead of CI gating, Ramp hosts evals on an internal platform where team members can add evals, tag cases for clustering, and run them on a recurring schedule (weekly or daily). A background agent reviews failures and regressions, and a Slack message flags regressions with a hypothesis of what broke.
hedged
AI Engineer · How Ramp Tests Its AI Bookkeeping Agent Before Customers Do - Ryan Stevens, Ramp
Recommendation · MoneyRun skill ablation like a leave-one-out feature analysis: compare a no-skill baseline (the raw model) against the model with each skill added to measure performance differences, assessing the incrementality of each skill. Use the results to build a candidate set of skills to delete or compress, saving context. Tool ablation can also be done, though it's more difficult.
AI Engineer · How Ramp Tests Its AI Bookkeeping Agent Before Customers Do - Ryan Stevens, Ramp
Recommendation · MoneyUse accuracy as a metric where partial credit counts (4 of 6 criteria right = 67%), and pass-at-K as an all-or-nothing metric for areas where the cost of error is very high, like accounting — if an accountant can't trust that one of the steps is right, they may not use your agent.
hedged
AI Engineer · How Ramp Tests Its AI Bookkeeping Agent Before Customers Do - Ryan Stevens, Ramp
Recommendation · MoneyThe creator recommends getting a still air box or a flow hood as the first item for your sterile lab, with a flow hood being the best way to protect from contamination.
Mountain View Mushrooms · Starting A Small Scale Mushroom Farm
Recommendation · FuelIf buying mushroom coffee for functional benefits, read up on the doses required to achieve those effects that have come out of proper human randomized control trials.
hedged
James Hoffmann · What's The Deal With Mushroom Coffee?
Anecdote · MoneyAfter three weeks of trying, Keiji decided to start over fresh because he no longer trusted Claude Code or Codex to do a full refactor of the system. He is a product manager and not about to write the code himself — he still uses a coding agent. He wrote a fresh set of specs with better goalposts, including rubrics, and started fresh with a coding agent.
AI Engineer · I Let Agents Refactor My Codebase for 3 Weeks. Then I Read the Code. — Keiji Kanazawa, Microsoft
Recommendation · MoneyPhilipp Schmid recommends creating credentials externally as write-only so they cannot be retrieved once written, only overridden, preventing leaks, and defining how and where each credential may be used, for example restricting a Slack credential to a certain domain or environment variable.
AI Engineer · 7 Things Your AI Coworker Needs - Philipp Schmid, Google DeepMind
Recommendation · MoneyDr. Sid Pednekar lays out three main vertical pillars of AI advantage: first, design the right problem; second, build shared capabilities; and third, compound the advantage.
hedged
AI Engineer · Your AI Isn't the Advantage. Your Judgment Is - Sid Pednekar
Recommendation · MoneySparsh Jain recommends building looping and eval infrastructure that, after sessions, cleans out things no longer important, prioritizes, and rearranges memory — which his team calls dreaming cycles.
AI Engineer · Your AI Agent Keeps Making the Same Mistakes - Sparsh Jain, CentralAgent AI
Recommendation · BodyJeff Cavaliere recommends altering the position of the dumbbell in your hand and leading with your thumbs rather than your pinkies when pressing. He explains that thumb-led pressing brings the hands closer together at the top, clearing about three to four more inches of range compared to pressing with knuckles facing the ceiling, and that these minor biomechanical tweaks are enough to give a better chest contraction.
ATHLEAN-X™ · Give Me 2 Dumbbells... I'll Build You a BIGGER Chest
Recommendation · BodyJeff Cavaliere recommends including the dumbbell pullover among chest exercises, but performed for chest rather than the traditional way. He explains that performing it traditionally with elbows flared out drives the movement with the lats, not the chest — to the point you wouldn't even look at it as a chest exercise. Instead, he advises getting your elbows closer to each other and driving the movement with the upper chest, and trying to turn your hands down and in (visualizing it as you pull forward) even though they can't move on the fixed dumbbell. He says this takes the lats out and drives the movement with upper chest adduction, and that it requires a little practice but is powerful for carving out the upper chest once you feel it.
ATHLEAN-X™ · Give Me 2 Dumbbells... I'll Build You a BIGGER Chest
Recommendation · BodyJeff Cavaliere advises that if you want to build upper chest but lack an incline bench, you can use a flat bench and dumbbells with an underhand grip — performing the underhand grip dumbbell bench press. He explains that upper chest fibers run down and out, so pressing should move low to high with the arms, and that getting the elbows down and underneath creates the opportunity to press from low to high. He claims this gives better upper chest activation, only slightly less than the incline bench itself.
ATHLEAN-X™ · Give Me 2 Dumbbells... I'll Build You a BIGGER Chest
Recommendation · BodyJeff Cavaliere advises keeping leg drive during the dumbbell bench press: with feet on the floor, push as if sliding your body up the bench and out as if extending the leg, not just down into the floor. He argues that skipping leg drive just because there is no J-hook target to push toward is a mistake, and that driving through the feet and pushing up maximizes the amount of weight you can lift.
ATHLEAN-X™ · Give Me 2 Dumbbells... I'll Build You a BIGGER Chest
Recommendation · BodyJeff Cavaliere says the chest fly is an exercise people don't load up often enough, and he does advocate loading up on it — but done as a floor fly. He explains the floor position has a built-in safety net: the floor protects the shoulders as the arms get down to the sides. On a bench, flies lack that safety net, forcing the use of light dumbbells, and loading heavy on a bench fly is dangerous. On the floor, you can use heavier dumbbells and take advantage of the eccentric contraction on the chest as you lower them to your sides.
hedged
ATHLEAN-X™ · Give Me 2 Dumbbells... I'll Build You a BIGGER Chest
Recommendation · MoneyRaymond Weitekamp recommends using an arbiter to gate which parallel decision-model outputs are allowed to act on and change the system state; the arbiter's job is to figure out where it has the confidence to allow an action through, given information across all channels.
hedged
AI Engineer · Your Agent Doesn't Need an LLM for Every Decision - Raymond Weitekamp, OpenProse
Recommendation · MoneyRaymond Weitekamp recommends a shared interface, which he calls 'the map', to let a decision model and a language model talk to each other; the decision model decides which map entries are the right thing to call, while the code underneath implements the function.
AI Engineer · Your Agent Doesn't Need an LLM for Every Decision - Raymond Weitekamp, OpenProse
Recommendation · MoneyRaymond Weitekamp recommends setting up the root of the system as a decision model that can choose to call a language model or agent as one possibility, calling this the more novel paradigm behind RDMs.
AI Engineer · Your Agent Doesn't Need an LLM for Every Decision - Raymond Weitekamp, OpenProse
Recommendation · MoneyThe Vercel COO advises building your developer surface area now — APIs, MCP servers, webhooks — so your product can be composed into agentic workflows without a human in the loop. Even if the work isn't simple, go build the developer surface areas now.
The Official SaaStr Podcast: SaaS | Founders | Investors · SaaStr 885: Going from 10 to 1 SDRs. How We Deployed AI Across our Sales Org with Vercel's COO
Recommendation · MindHuberman recommends three days a week of lifting weights and two to three days a week of some sort of cardio.
The a16z Show · Andrew Huberman’s Three Rules for Better Health and Focus