The last mile: reading, testing, and owning trade-offs
The last mile is where someone has to read the code, test the behavior, own the trade-offs, notice the missing requirements, and decide whether the thing deserves to be part of the system.
37,494 results · Most relevant
The last mile is where someone has to read the code, test the behavior, own the trade-offs, notice the missing requirements, and decide whether the thing deserves to be part of the system.
Not every line of code needs to be personally read, blessed, and understood in isolation — that has never really been the standard, and large systems have always contained more detail than any one person can hold in their head. But the logic has to be understood: the main flows, the boundaries, the data model, and product behaviors.
From the beginning, the team's hypothesis has been that one of the big bottlenecks in automated materials discovery is automatic characterization: mixing powders is easy, but without automated characterization and analysis you can't intelligently decide the next step. Mixing powders randomly yields little benefit — the value comes from automatic characterization and closing the loop with simulations, so you can try many things, make informed decisions, and plan the next day.
Rather than training on the final output of science, train on the process of doing science — the process data is the scarce, valuable resource. Build reasoning traces over entire scientific campaigns, not individual experiments: capture the full lineage of conversations, intuitions, lab executions, computations, and code, stitched together. The team says this type of data basically doesn't exist anywhere else and that they spend much of their time getting the full lineage of the scientific process into the model.
A classification algorithm can't be trained on all-positive samples — you need negative samples, which is why unpublished failed syntheses are a critical missing data source in materials science. The team deliberately seeks null results and learns from them: a null result means you intended to produce a structure and all evidence points to you not producing it, and sometimes the null reveals a previously unidentified structure. A string of negative results followed by a positive one is valuable 'process engineering' data — the iteration path to success is itself a training signal. They also run negative controls in materials experiments, because impurity phases can kill the property you're measuring, so controls are needed to distinguish real nulls from contamination.
As a product leader, the ideal time horizon to live in is two to three months out, not years. Predictions of the future years out are almost always wrong, so don't plan too far ahead. Tara keeps a personal doc of incorrectly predicted things she thought were going to happen and didn't, to remind herself of her inability to predict years out. If you build exactly for today, you're going to get left behind.
endorsed
Tara Sesha's bar for the AI era: aim products at where the models will be in two to three months, rather than being overly anchored in the present or overly futuristic and unusable. The product should be valuable, cared about by users, surprisingly great, in line with model capabilities, and get out of the model's way.
endorsed
Tara Sesha argues there are so many practical product considerations that the use case matters most in the single-vs-multi agent decision. Tactically, consider data access and permissions — e.g., switching between a service account or your account and how to manifest that understandably — and whether a single agent should have segmented memory, almost becoming a different agent when it joins a private Slack channel. Answer by going back to the specific use case and thinking through edge cases: how memory should behave, how it talks to each person, and whether it uses their credentials or a universal set.
endorsed
OpenAI thinks about ChatGPT as a platform and decides what functionality natively goes in versus what can be built by first-party or third-party developers as plugins or additional interfaces. The approach: structure capabilities in layers, exposing them as hooks developers can build cohesive experiences on; offer an array of different tools (e.g., a plugin to get all your meetings data and understand what to do next); and if pre-built or ecosystem tools aren't available for a task, computer use serves as a fallback layer so the user can still accomplish their task.
endorsed
Working with research is quite different from working with engineering — a realization from coming to OpenAI. To collaborate: come with very specific use cases and a clear idea of what users want and their goals; bring sample readings — understand literally what the session was, what it looked like, and why the model didn't do what it needed to do; and learn to write good evals, writing them as much as possible — the most important thing. The ideal deliverable to post-training is showing that with a specific prompt and skills, a desired outcome was achieved, so the capability can be post-trained into the model from the beginning. Collaboration starts with understanding it's a wholly different way of working, then getting good enough at bringing specific user data so researchers trust you,
endorsed
The appropriate planning cadence depends on understanding the pace of the market you're in — it's not the same for every business. Stripe's business dynamics look largely similar to how they looked before, so Stripe PMs can model bull/bear/base cases and plan many months out — and still do their annual planning — unlike OpenAI's business. Given the pace and dynamics of your market, determine at what cadence you should be planning.
endorsed
Tara Sesha describes OpenAI's quality bar: ship the product internally, have everyone use it, and check that it is retaining well, that people find something delightful or surprisingly great, that it unlocks a new use case of the models or does something novel, and that it is additive for users — unlocking additional value — before shipping externally.
endorsed
Marc Lore advises building the business by offering customers the value proposition you know you'll be able to offer at scale, even before you achieve scale — which means accepting losses in the meantime.
Marc Lore explains taking a page from the Walmart and Target playbook: use a loss-leader product to build customer relationships, sell it below cost permanently (diapers at a loss, even $1 boxes bought for $1.30), and profit from selling everything else. He stresses that losing money on the core product makes people uncomfortable — and that discomfort is exactly the differentiator between startups that succeed and those that don't.
Lore argues that people compromise their vision because it's too scary or risky. If you sell diapers at $1.40 when you buy at $1.30, nobody wants them and you have no value prop. Instead, build the business and offer the customer the value proposition you'll be able to offer at scale, even if you lose money in the meantime — he says that is in many cases the differentiator between a successful startup and one that doesn't make it.
A naive approach to sandbox management is handing out sandboxes to users round-robin without keeping track of which user gets which sandbox. Instead, keep track by user of which sandbox to get, so each user gets a dedicated sandbox they can rerun code in every time. The point is creating a map of user to sandbox ID.
Agents are microservices that can fail — a pod crash can take down the whole multi-agent system — so you must handle persistency and hydration; and without runtime-binding of thread ID, session ID, execution ID, and run ID (e.g., across LangGraph to Codex or Claude to CrewAI), agents cannot work as a group toward the same goal and task.
Hossein describes the improvement loop teams should build: observe failure modes in production along dimensions you define, grab those failure modes and create test cases to improve agent quality without introducing new regressions, then continue iterating over time. In the best systems you observe failure modes, log every input/output and every step of trace execution, analyze what went wrong, and create datasets of failure and success because you know the outcomes.
Dru Knox describes the core software factory workflow: all work starts as a trackable issue, gets sent to a headless agent running in a sandbox, the agent puts up a PR, and engineers engage via PR comments. Keeping human touchpoints inside PRs makes all agent interactions legible to tooling.
Dru Knox distinguishes autonomy (agents one-shotting problems) from automation (how much you can let agents run without review before accepting their solution) as distinct metrics — you can have high autonomy but low automation if you still review all code directly. He advises building up autonomy first before moving to automation, improving autonomy then automation, all while keeping quality constant. First hold quality constant, then aim to drive it up after that.
Choose sources → to get a library of your own and a daily brief.