Explore

Build variation into code-as-policy graphs: VLM in the loop and flexible failure handling

The interesting part is building variation into the code-as-policy graph: e.g., when detecting an object, use a VLM as part of the tool call. Put a VLM inside the loop so that while the coding graph is deterministic, there are points of variation that allow it to generalize. Handle failure with flexible responses: define how to check that something failed and how to potentially do something else depending on the failure.

hedged

Y Combinator · Robot-Use Agents: Why General-Purpose Models May Win in Robotics

Pipeline existing compiled reports into the database and expose them as agent tools

Jeff Smith recommends piping your team's existing compiled reports (SharePoint, network drives, folders of reports) into the database and/or MCP server and presenting them as tools or APIs the agent can call. Instead of having the agent task an LLM to generate a query, present the available reports to the agent and ask which it would like to look at.

AI Engineer · Your AI Agent Wants Your Data. Here's How to Lock It Down - Jeff Smith, Oracle

Know who is looking at the data before returning it; use SSO instead of database auth

Jeff Smith advises knowing who is looking at the information in the database before returning any data back to them — the information he is allowed to see is much different than what his colleague Tara can see. He recommends not using database authentication to let the agent go in and do stuff, and instead piping users through the organization's single sign-on (like Azure Enter ID), which returns a list of scopes, which returns a list of roles that describe your role in the organization.

AI Engineer · Your AI Agent Wants Your Data. Here's How to Lock It Down - Jeff Smith, Oracle

Don't let LLMs act on production databases; use advice-then-action and safer environments

Jeff Smith advises not to trust an LLM to drop an index, build an index, or do anything in the production database — have it give advice and then action that advice yourself. He also advises not letting the LLM or agent into production at all if you don't have to, and instead using a test environment that mirrors production to investigate performance issues with an agent.

AI Engineer · Your AI Agent Wants Your Data. Here's How to Lock It Down - Jeff Smith, Oracle

Build out semantic and ontology layers for agent accuracy

Teams should build out their semantic layer and their ontology layer because it becomes very important for agent accuracy; a 5% margin of error becomes very critical when working with the finance sector in any mission-critical workload, where even minimal errors can mean a 100 million fine for handling fraudulent transactions or not.

AI Engineer · Everyone Has the Same AI Models. Here's What Sets Agents Apart - Richmond Alake, Oracle

Examine each system layer for replacement by a decision model

When designing an AI system, examine each layer and consider whether it could be replaced by a decision model; today you can have models all through your retrieval pipeline.

AI Engineer · Everyone Has the Same AI Models. Here's What Sets Agents Apart - Richmond Alake, Oracle

Treat memory as data, write down entities and definitions, push entitlements into the data layer, and capture traces

Richmond Alake advises: treat memory as data and don't look at it as different things; write down your entities and definitions; push entitlements into the data layer; and capture memory traces because they become data for fine-tuning.

AI Engineer · Everyone Has the Same AI Models. Here's What Sets Agents Apart - Richmond Alake, Oracle

Bring models into the agent memory system: decision models, embeddings, re-rankers

Within an agent memory system, you can bring in models to improve the system itself, such as a decision model, an embedding model, and a re-ranker; if you own your own memory, you can fine-tune open source models used in your agent memory system.

AI Engineer · Everyone Has the Same AI Models. Here's What Sets Agents Apart - Richmond Alake, Oracle

Capture memory traces, format them, and retrain models for quality and latency gains

Once you understand your intelligence plane, you capture your memory traces, format them effectively, and retrain the models; retraining models on captured memory traces yields benefits with improvements in quality and, in some cases, reduction in latency.

latency reduction holds only 'in some cases'

AI Engineer · Everyone Has the Same AI Models. Here's What Sets Agents Apart - Richmond Alake, Oracle

Test a lot and be data-driven when building AI features

Supriya Subudhi recommends testing a lot and being data-driven: test how different models impact different datasets, compare cost differences, and retest versions of prompts and feedback quickly, using eval setups to enable this.

AI Engineer · 78% Wanted This AI Feature. Almost Nobody Used It - Supriya Subudhi, Bumble

Two approaches to global compliance: build within all restrictions, or launch in your highest-impact market only

Supriya Subudhi outlines two approaches to global compliance: decide upfront to ship worldwide and build within all restrictions from the start, or launch the best possible product in your highest-impact market (e.g., the US) and accept not launching elsewhere.

hedged

AI Engineer · 78% Wanted This AI Feature. Almost Nobody Used It - Supriya Subudhi, Bumble

Automate tedious system-of-record workflows: procurement and reconciliation

The guest recommends automating tedious system-of-record workflows — ERPs, CRMs, procurement, finance reconciliation — as the highest-value enterprise use cases: having teams fill a spreadsheet and letting the agent transfer it into procurement software, finalize line items, and reach out to vendors; and having the agent look through company ledgers, reconcile them, and flag anything interesting for a person, since many financial software products lack APIs for quickly pulling data.

The a16z Show · OpenAI: When AI Agents Use Computers Faster Than You

Set up LLM-as-a-judge scoring (online evals) over all agent runs

Zach Lloyd recommends setting up a scoring mechanism using LLM-as-a-judge that reviews all agent runs and evaluates whether they are going well — what he calls online evals or 'scores', where a set of agents looks at all agent runs going through the system and grades them. He argues that without scoring you cannot detect failure modes or know whether runs are going well, and that it is much more efficient to have agents grade scorer output than to grade manually as a person.

AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp

Set up a self-improvement loop where agents analyze failure modes and propose PRs to the factory definition

Zach Lloyd recommends setting up a self-improvement loop where agents look at failure modes, aggregate them, look for patterns, and come up with ideas to improve the underlying factory definition — closing the loop by having the agent propose a PR to the factory definition when it identifies a recurring failure mode. He illustrates the agent's framing as 'I saw this fail 37 times' and 'we can actually fix this with a PR to the factory definition'.

AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp

Create your own benchmarks from real coding agent tasks

Zach Lloyd recommends creating your own benchmarks by taking all of your real coding agent tasks and trying them against different models, measuring how it goes. As a starting approach, he recommends building a benchmark set by hand, picking a set of small, medium, and hard representative tasks. Alternatively, you can ask the factory itself to build the benchmark set, since it is aware of all past sessions and everything is stored in code — the factory can look at prior sessions and identify interesting ones. You can also take any past run from your system and add it to a benchmark suite, and curate a benchmark of representative tasks from your own factory to measure model performance on your own data. He gives the example of building a suite of front-end tasks to answer which model isbest

AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp

Set up online evals (scores) with LLM-as-a-judge

Zach Lloyd recommends setting up scores — online evals — where a set of agents looks at all the runs going through your system and uses LLM-as-a-judge to figure out whether the runs are going well. At Warp, they have a set of agents looking at all of these runs, and a scoring mechanism that evaluates them using LLM as a judge.

AI Engineer · How to Build a Software Factory That Improves Itself - Zach Lloyd, Warp

Frontier models for critical tasks, smaller models for the long tail of tokens

Daksh Gupta recommends using frontier models for the most critical tasks (analyzing bugs, applying the review process, submission) and smaller models for the long tail of tokens (exploratory, file reads).

AI Engineer · How to Build an AI Code Review Agent - Daksh Gupta, Greptile

Give agents access to JIRA, Linear, and docs stores

Daksh Gupta recommends giving the agent access to JIRA, Linear, and a docs store like Confluence or Notion, claiming you can get remarkably far with just these two types of context: docs and tickets.

AI Engineer · How to Build an AI Code Review Agent - Daksh Gupta, Greptile

Use model inversion in your code review process

Daksh Gupta highly recommends using some type of model inversion in your code review process: if you used Claude Code to write the code, use a GPT model to review it, and vice versa.

AI Engineer · How to Build an AI Code Review Agent - Daksh Gupta, Greptile

Use Multigress for Postgres sharding without changing queries

Multigress was ground-up rewritten so you don't have to change any queries when you shard; it was stress-tested with 1,000 sequential failovers and clobbering tests with zero data loss before release.

Latent Space · Supabase’s Plan for a Billion AI-Built Apps