Creator perspective · MoneyTanmay Sah explains that each core dimension is divided into sub-dimensions, forming a framework of 13 sub-dimensions and 3 core questions. Technical strength includes data quality, task design fit, predictive validity/outcome analysis, fairness checks, failure-mode analysis, and explainability to regulators. Deployment includes decision authority given to the AI, human-in-the-loop effectiveness, exposure control (who uses the system), and monitoring controls. Business impact includes financial impact, legal consequences, and scale (how many users are affected).
one atom flagged possible_numeric_error on the count of technical-strength sub-dimensions; framework counts are as stated by the guest
AI Engineer · 1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life
Principle · MoneyThe guest endorses 'quantity breeds quality' in creative volume, but says there needs to be a quality standard when approving. Every submitted video is watched start to finish (now by his team), and if it doesn't genuinely have all the parts and components of a great ad that could do well, it's denied. A quality ad must have a hook, body, call to action, and unique value propositions, and must meet production quality standards: no echoing, good lighting, good editing — if it doesn't have any of that, it's denied; if it's great, it's approved.
Foundr · The NEW Creator Playbook that Took Me from $80K Debt to $250K/Day | Trybe
Principle · MoneyThe guest shares his saying: it's a lot easier to teach an actor how to act like an athlete than it is to teach an athlete to act like an actor. Instead of going for someone you think could be good, you have to find creators who already know how to act — who have already started making content and know the basics. TikTok is still an amazing place to find amazing creators, even today with Tribe's 280,000 creators. Don't go for creators making a ton of money — go for ones making a couple thousand a month, so they're excited and your brand can be the life-changing opportunity for them. After finding the right creators, sell them on why your brand is the best opportunity.
Foundr · The NEW Creator Playbook that Took Me from $80K Debt to $250K/Day | Trybe
Creator perspective · MoneyTechnique 4: change things that should not affect the result — the supplier's name, the supplier's address, reordering clauses — and verify the number of flags stays the same; also run the contract again and again to check output consistency. In the invariance technique, everything can be checked using code — no LLM-as-judge needed; the team stays away from LLM-as-judge as much as possible because it's a very hard task (as the previous speaker showed).
AI Engineer · How to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
Creator perspective · MoneyTechnique 3: take an example of a contract that has been signed, remove a clause, and the agent is supposed to say 'Hey, there's something missing here.' The missing-clause technique is very hard because different suppliers have different templates and content, so knowing what should be in which contract requires thinking; these were mixed with existing contracts that should not show any flags. The missing-clause approach is the most powerful approach they had: it generates a lot of contracts, and it revealed the agents were missing most of the clauses that had been removed — a very strong benchmark. When an agent is missing most of the clauses that have been removed, that finding itself is a very strong benchmark result.
AI Engineer · How to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
Creator perspective · MoneyTechnique 5: turn an example into many using the concept of templates — it's the most powerful technique but very hard; Air Canada's experts have different kinds of templates with blanks that should be filled by the expert. Use expert templates with blanks as the basis for generating synthetic test contracts. Generate an array of fill values for each template blank instead of filling blanks manually, enabling generation of many contracts. Deliberately generate some tuples that break rules and some that do not, so you know exactly what should be flagged as a breach. Check template-generated contract evaluations with code, without involving any LLM. Reuse templates that experts already use in their normal workflow to generate test contracts, reducing friction. Expect roughly a four-week
AI Engineer · How to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
Creator perspective · MoneyTechnique 2: start from rules instead of creating an agent that provides flags — take a known rule (e.g., a cap above $1 million that every Air Canada contract should follow), create a clause that breaks the rule, generate the contract around it, and check the agent finds the flag back. The rule-based technique's problem is that you need to know the rules; coverage of everything cannot be guaranteed, so the team also came up with their own rules. Using rules from experts and documentation plus their own rules, the team generated 275 tests where they know the test can run and know the answer. When you know a specific rule, create a clause that breaks the rule, generate the contract around it, and test whether the agent finds the flag. When you have limited access to domain experts, use
AI Engineer · How to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
Report · MoneyThe biggest impact of AI on the speaker's content is especially with captions — their captions now are so different from before because the AI can tell them which hook types aren't working for interviews and suggest trying different styles, then report which performed well and which didn't. To make A/B testing more apples-to-apples: take a longer video, create several cut-downs, test different caption experiments across them, and use the API to compare engagement, since the base content is more similar.
Personal experience plus a recommended method.
AI Engineer · The AI Content System Behind 200M Views — Sarah Chieng, Cerebras & Cova Lee, OpenAI
Principle · MoneyAdit Abraham argues that if you have a harness where the agent is not only extracting information but also correcting itself and reviewing its outputs, you can achieve higher accuracy than human annotation — he calls this the first time agents can be better than humans at structured extraction (via Reducto's Deep Extract). Human-annotated ground truth often contains errors (missing content, dropped rows, lazy partial fills) that go undetected because there's no comparison point.
The a16z Show · The Missing Infrastructure for AI Agents: Documents | Reducto’s Adit Abraham
Principle · MoneyThe speaker recommends comparing a creative's organic views to your account's norm: if a post performs well relative to your norm, that is the creative to put ad money behind, because content cannot get views unless the creative is relevant. He claims this should save an extraordinary amount of money, noting many people waste money on social ads whose creative will not convert.
GaryVee · Everything You Need to Know About Social Media in October 2026
Principle · MoneyGaryVee advises leaders to empower people underneath them: instead of suppressing a direct report with bigger natural strengths in parts of the job, become an accelerator of them. He frames suppression as an anti-pattern and suggests auditing whether you are a suppressor or an accelerator.
GaryVee · The Leadership Mistake I Made for 20 Years
Principle · MoneyGaryVee describes his practice of 'kind candor' — putting the word kind in front of candor — which he says completely reactivated his organization. He observed his executives using radical candor as a weapon to suppress people or be jerks with their words, and advises not using candor as an excuse to be a jerk. He notes most companies need kind candor for the kind part, while his organization weirdly needed it for the candor part.
GaryVee · The Leadership Mistake I Made for 20 Years
Principle · MoneyGaryVee describes his operating model: he works for the 2,800 people his company employs — they don't work for him — and he asks how he can bring them value, believing that serving their value first gets him everything he wants.
GaryVee · The Leadership Mistake I Made for 20 Years
Principle · MoneyFor an intelligence producer, rent is one part of the cost. The operator still has to turn compute cost into work the customer can actually use. To compute full cost per useful output, count everything spent to deliver the service and divide by the output the customer can actually use. Operating capability affects both sides of that equation: what you spend and how much useful work you can deliver.
AI Engineer · Same GPU, 50% More Output: Why the Operator Matters — Oliver Kennon, SemiAnalysis
PrincipleOwen models RSI progress as a ratio: capability improvement rate equals performance (numerator) divided by cost/difficulty (denominator). He stresses always accounting for the denominator — when evaluating AI progress claims, divide capability gains by capital raised, since model release times may be shrinking but improvement normalized by investment may not be.
The a16z Show · The Race to Recursive Self-Improving AI
Principle · MoneyThe host explains that once you save a search, every new person fitting the criteria automatically appears in it, so you never have to recreate the search — you just respond to new prospects.
30 Minutes to President's Club | No-Nonsense Sales · #616: How I Use LinkedIn Sales Navigator to Find Warm Leads (Crash Course)
Principle · MoneyWhen evaluating an acquisition-prediction system against the live web, searching 'now' leaks future knowledge (the acquisition is already visible), producing a false 100% true positive rate. The fix: use Snapshot to set the snapshot date to just before the outcome was announced, so the only information informing the system doesn't leak the exact confirmation of that outcome. For example, when designing a system to understand company acquisitions, you might search one day, one week, or one month before the acquisition to check whether the search would have found the relevant signals.
AI Engineer · Backtest Your Agent Against Yesterday's Web — Ryan Ahern, Exa
PrincipleWhen a pilot tests poorly with the general audience but a small group loves it, treat it like a startup: aim to be a 9.9 for a very small group of people rather than a 7 for everyone, then get those superfans to tell their friends.
Acquired · Seinfeld
Principle · Money'Status comes from records, not model pros.' Show me the receipts — the actual tangible proof that it did what was expected. Don't just accept 'yeah, it passes, it looks good' from an agent — there's nothing to back that up; demand actual screenshots, videos, test results, and smoke tests.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Principle · MoneyIf your web app does not already have a bespoke CLI that allows any agent to fully flex the functionality you care about verifying, you'll have a much rougher time and an unpaved path using prompts alone. Give each agent the ability to say: yes, I actually built that feature, it is actually performant, it didn't cause any errors, and the application looks correct.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS