Creator perspective · MoneyThe teacher agent evaluates the doer by reading JSON trace files created for every iteration, uses a Claude Agent SDK loop with a system prompt and the previous strategy to produce a better strategy, and writes new learnings back into strategy.md. The strategy.md file is injected into the doer's loop at every iteration, and a skill.md file accumulates all hacks, tricks, and learnings from previous runs. The teacher also decides when to stop the research loop rather than a fixed iteration limit.
AI Engineer · Hill-Climbing Skills: Improve Agents Without Changing the Model — Shubhankar Srivastava, Browserbase
Creator perspective · MoneyAll fundamental principles of the system are loops: when you do something, you always verify and criticize yourself. Verification is much easier than generation or general action, which is why model-based verification catches errors. For form filling, take the filled form, reverse it, and ask the model whether it makes sense — most of the time the model recognizes something wrong. If verification finds something wrong, take the results back and ask the agent to do the same task again; this improves performance a lot. Ang Li analogizes this to friends criticizing his ideas so he can reason with less bias.
AI Engineer · The Autonomous Computer: Infrastructure for Computer Use Agents — Ang Li, Simular
Report · MoneyThor describes the Live API as a low-latency, real-time voice and vision API. It is a stateful WebSocket API: you create a session, send real-time text, audio, and video frames to the model over the WebSocket, and the model responds in real time with audio chunks you can stream in your application, along with the transcript of what was said. Audio transcription is also available as a feature.
AI Engineer · Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff
Creator perspectiveLLM-as-a-judge only reads input/output against a fixed rubric and catches known issues. Harness-as-a-judge moves evaluation to an agentic flow that can call tools, connect to external systems, and catch unknown-unknowns via trace analysis.
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
Report · MoneyNathawani warns that continually prompting an LLM in a loop (e.g., 'give me a product, a review, and a customer profile') leads to repetition after 10-20 tries and lacks diversity, correlations, multimodality, steerability, and validation.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Creator perspective · MoneyThe speaker advises asking many small questions rather than one big question when building auditable semantic analysis on top of deterministic analysis. Throwing one gigantic prompt at a model with a large SQL file causes hallucination and invented information. Instead, parse the SQL and ask about one column at a time with its type and comments, and weigh a semantic question by giving more prompt input to that single focus question.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyYoni Michael describes a safety practice: before writing SQL to change a model's grain, walk downstream in the graph and ask each downstream dependency node whether the change breaks it.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyYoni Michael describes how a business analyst with a wrong dashboard number should be able to trace the issue back through multiple layers of the data pipeline to find the root cause — for example looking back three layers to find a wrong select, tracing back to a specific commit or deployment that landed recently to identify when and why it broke, and checking for errorful runs that populated the data to understand when the problem occurred.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyThe first area where coding agents fail when applied to data is impact analysis ('what breaks if I make this change?'); the second is join recommendations — whether a join is actually safe; the third is safe transformation logic changes — whether an edit preserves the meaning of the metric.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyYoni Michael argues you must meet the model halfway: adjust tool design to match its training patterns, use the same tool structure and response format as ripgrep for graph tool results, and name tools carefully (a tool named 'search' triggers multi-keyword search with no training).
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyA data context layer can silently replicate every single row when a join goes from a one-to-n relationship, so the agent must understand which join scenarios will result before acting; choosing a join path based on column names alone can cause a fan-out, a 7X inflation in rows, or silently drop 12% of rows downstream.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyChanging a model's grain from monthly to weekly causes a monthly ARR measure to be repeated across weekly rows, so downstream ARR sums show approximately 4X the accurate value; before writing SQL to change grain, an agent should walk downstream in the graph and ask each dependency whether the change breaks the grain represented there.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyWhen an agent tidies up a metric and removes a predicate, the SQL still compiles but the predicate may be a critical part of the business definition — e.g., removing a 'paid' predicate silently redefines the metric to include trial churn across all downstream dashboards.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspectiveOmar argues that parsing complex documents like the Epstein cover sheet with a general-purpose LLM (ChatGPT) yields inaccurate results because layouts are hard to understand and reading order is subjective.
AI Engineer · Parsing the Epstein Files for JMail — Palak Agarwal & Omar Alhait, Reducto
Principle · MoneyPeople need to realize you're not ranking a model — you are benchmarking a specific configuration. If you change the dataset, it's a new benchmark. Behind the model sits the inference engine and whatever happens on the endpoint, which you often can't control. If the provider updates the inference software, results can change again, and you have no way to see that unless it's your own inference on your own system. The agent harness version matters too: testing Claude Code or Codex next week after an update is a different version than today. Even sandbox providers can change something in their backend without you noticing, and it could affect your evaluation results. Change any part of the configuration — dataset, model, endpoint, harness, sandbox, or settings — and it's a different.
stated
AI Engineer · From Zero to Leaderboard: Agent Evaluation — Wolfram Ravenwolf, Weights & Biases
Creator perspective · MoneyRemy describes an internal AI deployment as four components: the LLM(s), the data connector layer (MCP for many, but really CLI, APIs, or any way of getting data, even reading a file), the interface, and the sandbox/environment where the agent runs. It's important to be able to separate these components and make a choice at any given layer at any given time, because the right choice differs across teams — an engineering team won't work via a Slack thread, but a marketing team might.
AI Engineer · An AI Future Without the Lock-In — Remy Guercio, Tailscale
Creator perspective · MoneyPeter Werry explains that behind the scenes at Unblocked they use a lot of static schemas because they want correctness absolutely nailed down; the schema is static, tied to their code base, and they control it. For a completely unknown schema, as documents get added the schema could update, so your choice is either to rerun schema generation on a sampling interval or use a static approach.
AI Engineer · Beyond RAG: Build a Relational Context Engine from Scratch — Peter Werry, Unblocked
Creator perspective · MoneyPeter Werry argues that full-text search identifies keywords as one component of a hybrid retrieval system, but you still need a mechanism for semantic similarity search because ideas and sentences are encoded that aren't necessarily textual matches.
AI Engineer · Beyond RAG: Build a Relational Context Engine from Scratch — Peter Werry, Unblocked
Report · MoneyPeter Werry describes an anti-pattern where generated queries using where or function clauses can literally run JavaScript on the server, which means this is now a prompt injection point.
AI Engineer · Beyond RAG: Build a Relational Context Engine from Scratch — Peter Werry, Unblocked
Principle · MoneyIf you can stay within a second and a half end to end, the conversation still holds, but you're right on the edge. If you can get closer to 600 milliseconds, the conversation starts to feel really human. The question is never 'how fast can I go' but 'what is the most human-like agent that I can fit inside this window' — and the question 'what models should I pick' was never actually answerable.
AI Engineer · Latency Is a Budget. Humanlike Is the Goal. — Jesse Hall, LiveKit