Recommendation · MoneyTo shift from product to harness engineering, find the judgment your team always asks the same person about and make it explicit — don't just write a skill, which is only one implementation.
AI Engineer · Scale the Judgment, Not the Model — Andrew Orobator, Reddit
Method · MoneyTenev describes how agentic trading accounts are set up and operate: create a separate segregated account and move money into it, keep trade approvals on by default until you want the strategy to run autonomously, then run a daily agent loop that executes your strategy, rebalances the portfolio, and looks for opportunities — optionally connecting alternative data sources like satellite data via the app store. The agent is sandboxed and cannot access other accounts or move money outside the segregated account.
Method, not a verified protocol.
No Priors: Artificial Intelligence | Technology | Startups · Tokenization, Agentic Trading, Harmonic: Robinhood's Mission to Expand Ownership with CEO Vlad Tenev
Recommendation · MoneyThe speaker says it is extremely important to keep testing regression cases, known tasks, and past failures, and that fresh challenge cases at all points go back to setting the most ambitious target. You have to keep pushing your model and make sure it is challenged and has cases where it is not able to perform to expectations.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker identifies adversarial tests as the third part of a holistic evaluation system: find where you can break the workflow and put it into challenging situations. He advises choosing a test for the failures it can reveal — the idea of a test that can fight back.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker identifies two ways to get to an eval that is as holistic as possible: consistently human review your process, and ensure that your customer feedback is part of your eval pipeline. He explains these catch the last 5% edge cases that a model or an existing eval framework might not catch.
hedged
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker advises starting with the job and asking what must happen, what must never happen, and when it should stop, then extending these core questions into other WH questions and first principles to think about the most ambitious outcome. He also endorses aiming at the job you need, not the score you can get.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyFor high-stake applications, the speaker's first recommendation is to define the most ambitious success before designing any test. He suggests assuming a world of AGI: what is your AI agent going to do, or what is the best outcome you can get out of it? Setting this most ambitious goal gives you the scope of what you can go ahead and design further.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker advises continuing human review and making sure the results make sense, and discourages relying on your agents to be the reviewer of your eval at any point, ever. He notes you can fast track your eval process with agents, but make sure you're doing human review.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker advises making automated checks and evals part of your CI pipeline, with eval questions asked to your agent every time they go through a production deployment process. He explains this ensures you're not just taking one eval score at the time you're deciding the model.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker advises continuously testing the workflow at every part of the workflow, discourages assuming things in the workflow or assuming an input and output to be the outcome of anything, and says to make sure your tests are designed for what that element of the workflow does. He notes the objective in that workflow process is to know your tool call ability and tool call accuracy at that point.
AI Engineer · If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com
Recommendation · MoneyThe speaker advises reading real AI outputs, which evals make possible: setting up the right eval tools let Bumble see what real users were receiving and whether the prompt was doing a good job. Bumble's success criteria changed from overall green-versus-red results to examining the long tail of data — whether different segments of users receive feedback well, including whether women respond the same way as men and whether users in different countries respond the same way, since language generated in French may differ from English.
maybe the language generated in French is different from the language generated in English
AI Engineer · 78% Wanted This AI Feature. Almost Nobody Used It - Supriya Subudhi, Bumble
Recommendation · MoneyThe guest recommends enterprises use deterministic controls — sandboxes and human-in-the-loop approval gates — to feel safe deploying computer-use agents, even when the model itself is highly aligned.
The a16z Show · OpenAI: When AI Agents Use Computers Faster Than You
Recommendation · MoneyMasaba Gupta advises making everything on brand and never going off brand, keeping the storytelling and language the same across all brand touch points — she calls it the brand's 'body language'. She stresses that the story created in the head office must permeate all layers of the brand: retail, e-commerce, social media, communication, and all touch points, and that when sitting down with each team, whether creative or retail, she delivers a singular message.
Figma · Config India 2026: What makes the House of Masaba feel like Masaba
Recommendation · MoneyZach Lloyd recommends defining your software factory in code, with a strict definition covering the workflows, the skills, the MCPs, and where these run. Version-controlling the factory definition lets you test different versions of your factory and lets coding agents update your factory. He suggests having an actual code repo that sits behind the factory, where you can see a definition of the entire factory at any given point in time, including everything that goes into automating development: agent definitions, automations, benchmarks, where agents run, the skills, and the webhooks — all stored in a repo that allows for automation.
AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp
Recommendation · MoneyZach Lloyd recommends setting up a self-improvement loop where agents look at failure modes, aggregate them, look for patterns, and come up with ideas to improve the factory definition. Agents can propose fixes to failure modes via a PR to the factory definition. One example fix is having agents re-read matching repository skills before implementing. He frames closing the loop — agents identifying problems and then fixing them — as a core concept for improving development.
AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp
Recommendation · MoneyZach Lloyd recommends storing the entire factory definition in a code repo that allows for automation, including agent definitions, automations, benchmarks, where agents run, skills, and webhooks. He argues that defining the factory in code and version-controlling it enables testing different versions of the factory and lets coding agents update it — which is what enables the factory to improve itself.
AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp
Recommendation · MoneyStandardize evaluation infrastructure (reusable frameworks, metrics, judges calibrated with human feedback); automate feedback loops (continuous trace collection, recurring failure detection, root cause analysis); scale automated optimization (prompt optimization, agent reflection, repeatable candidate testing); validation and trust (holdout evaluations, regression testing); and democratize RSI with reusable components and self-service info.
AI Engineer · An Agent That Fixes Its Own Mistakes - Saloni Gandhi & Ziwei Guo, LinkedIn
Recommendation · MoneyStart with prompt updates as the easiest and most straightforward way to iterate; then optimize the structural context of the AI system; then expand scope to automatically design the entire workflow (meta information design); then optimize the harness code itself; and finally, the most complex step, optimize the optimizer code itself.
AI Engineer · An Agent That Fixes Its Own Mistakes - Saloni Gandhi & Ziwei Guo, LinkedIn
Method · MoneyThe RSI process proceeds in ordered steps: (1) run the agent and generate samples — for the career agent, given a query, generate output and log the process in traces (e.g., LangSmith traces); the agent trajectories and traces generated are the material used for optimization. (2) Run the traces through the evaluation engine (e.g., the LLM judge) to get judge output telling where the system is good and where it is not, to guide future iteration. (3) Agent reflection (also called root causing): based on the judge output, understand the root cause of the bad output. (4) Based on the identified root cause, generate an edits proposal — currently the edits are prompts, but in future they can be new workflow designs, harness code, or optimizer code. (5) Validate the edits, because some edits can
Method, not a verified protocol.
AI Engineer · An Agent That Fixes Its Own Mistakes - Saloni Gandhi & Ziwei Guo, LinkedIn
Recommendation · MoneyDustin advises limiting SaaS demos to no more than three features — never going over three — and picking out the features you're going to demo in advance. When you have a large feature set and limited demo time (30-40 minutes), he recommends scoping down to drive home key points. He cautions against letting prospects drive the feature selection, because you end up so in the weeds with your technical baby that it derails the demo.
MicroConf · SaaS Sales Demos: Why Getting a No Is Better Than a Yes