36,594 results · Most relevant
Set up agents by creating a blueprint that acts as a ceiling of access for each agent (e.g., one for the read-only agent, one for the fund agent). When a user invokes an agent, the scopes in the issued agent token should be the intersection of the user's permissions and the scopes in the agent blueprint's ceiling. The agent then calls the application's API using a scoped agent token.
Isabelle Williams explains that if the agent's provider has already verified the user's identity, the agent provider can tell your application who the user is instead of requiring manual signup. When an agent asks the user for consent to share their identity and email with your application, the agent requests an ID-JAG from the agent provider. The agent provider verifies the user's identity (e.g., verified email) before issuing the ID-JAG, so the application can trust the assertion. Your authorization server verifies the ID-JAG signature before issuing an access token to the agent, and your application decides whether to create an account for the ID-JAG user, link it to an existing account, and what scopes to grant — it is completely in control.
Isabelle Williams recommends hosting a markdown file called auth.md with your application that describes how agents can register for accounts, discover how to get started, create an account, and call your API. The file starts with standard OAuth parlance: finding the protected resource, the API, the MCP server, and the OAuth authorization server. She emphasizes that the application itself, not the auth.md file, is what enforces auth at the API.
The agent discovers how to authenticate with your API using standard OAuth parlance, asks the user for consent to share their identity and email with the application, and if the user consents, goes to the agent provider to get an IDJAG. The agent sends the IDJAG to your API's authorization server, which verifies that it is legitimate. Your app is completely in control of deciding whether to create an account, link to an existing account, and what access to give back to the agent.
In the user claim flow, an agent can take anonymous actions at your API and receive an access token plus a claim token. The agent gives the user a link and a one-time code; the user signs up, signs in, and enters the code to claim the agent's actions under their account. Using OAuth, your authorization server can then issue an access token on behalf of that user to the agent.
The team gives each LLM judge around a thousand words of examples, instructions, and directions per metric, plus the script to analyze, their own script as guidance, and the brief and research — a 14,000-word prompt total. The judge first categorizes between six categories, then gives a rating within that category, and must return a score with a justification and a specific supporting example. The judge analyzes writing quality only (not length) and doesn't know which model wrote the script.
Bouchard's team measures five things for each generated script: tone (does it sound like you and adapt to a specific voice), pure writing quality (hooks, emotional transitions, story aspects), instruction-following (expected length, format, images in the right place), research accuracy (hallucination check, correct use of research), and slop (EQBench slop score plus their own measurements).
Gary Vaynerchuk describes his '1585' system: involve yourself in the first 15% of a project to ideate and contribute strategy, let the team completely own the next 80%, then come back in for the final 5% to add a finishing touch.
When a GitHub issue comes in, write a Prose program for how the agent will try to address it — as complex or simple as you want, including multi-agent workflows, specific skills, or specific models. Use pull requests for event-driven adversarial review of agent-built code, letting agents receive back pressure from the app itself. From the trigger you triage workflows, so you can build out code factories from creating standing jobs of the Prose programs. The agent running cannot consider its job complete until its developer agents have taken the reviewer feedback into account.
Al Chen endorses a simple disaster recovery pattern for ClickHouse: run ClickHouse in a primary region, a job that backs up to a bucket, and another job that restores that bucket to a DR region, where you might have multiple pods with replicas of the data. The procedure relies on ClickHouse backup, an open source project maintained by Altinity (itself a competitor to ClickHouse for managing data failover), which shares logs and a status table of whether backup and restore happened. As an extra step, check row counts between tables and run a check that replicas within the region have converged.
endorsed
Al Chen endorses a flow for handling judge reasoning safely: let the judge read the raw trace data but do not persist the judge's reasoning; the score always persists through the UI because you want it for evals. In application code, a check tells the agent whether the record contains sensitive information — if it does, suppress that reasoning from the UI. Most banks he works with want to see the LLM-as-judge's reasoning on sensitive information, so an extra upgrade is a second LLM-as-judge whose input is redacted of sensitive information, giving confidence its output is clean and can show up in the UI.
endorsed
Al Chen describes a locally coherent method for verifying CVE remediation with Harbor: instruct the agent that version 2.6.2 of a package supposedly remediates the CVE and ask it to write an exploit based on the older version to see if the new version still has that exploit; then have the agent write a test to see if that exploit actually passed; and run many Docker containers or Modal or Daytona sandboxes in parallel to get many trials of how the agent performed.
described
A method for automated quality gates: anchor experiments on datasets, set up CI so any agent or prompt change triggers a GitHub CI job running evals against the dataset, define minimum score thresholds per evaluator, and treat failing scores as a block on production promotion.
A method for assembling evaluation datasets: create use-case-specific datasets (including red-teaming sets for prompt injection), source items from team spreadsheets, production traces, and curated good examples, and keep domain experts in the loop defining golden datasets.
A method for scoring agent outputs: extract common categories from domain expert feedback to define dimensions of quality, build a library of evaluators that automatically score incoming traces (optionally in real time from the client application), and blend LLM-as-a-judge with deterministic checks — relying 100% on LLM judges risks inaccurate outputs.
A method for getting domain expert feedback: build an annotation queue routing agent outputs to experts, give them simple scores (yes/no, 1-10), allow free-text comments, and keep the process very simple since experts have day jobs — ask for about an hour of their time on a curated batch.
Set up a Sales Navigator search for executives who leave your current customers and join accounts in your territory, even if you don't know them, because they've likely had a good experience with your company. Move fast on these alerts — speed to lead matters before competitors reach out — and existing customers have higher odds of taking you with them to their new company.
Every time a deal closes, save all stakeholders — procurement, legal, champions, moles, skeptics — into a LinkedIn list (not a saved search) so you get alerted when and where they change jobs. The list tracks when a person leaves a company regardless of destination, not just moves into your territory.
Block time on the calendar to work alerts or it won't get done — a daily triage in the morning (when it's quiet, before Slack gets busy) or before closing out the day. During the block, check alert reports and take action on the results. Setting up alerts without a way to work them leads to forgetting, no booked meetings, and sadness.
When a profile viewer is tied to an open opportunity, reach out to capitalize on the data, create a conversation, and possibly accelerate or expand the opportunity — a fantastic way to multi-thread — without calling attention to their tie to the deal. For an unknown viewer with no existing opportunity, send a connection request framing the specific challenge you solve for their industry peers, or spark dialogue by asking what made them swing by. Don't sit quietly waiting for them to go inbound.
Use saved training trajectories and a controlled intervention to explain a failure, separating equilibrium theory from observed empirical stability.
Demonstrate all 3 repaired bugs and defend why a passed gradient check is necessary but not sufficient for generalization.
Use the tractable fixture and posterior predictive results to separate approximation, sampling and model errors; justify one revised modeling assumption.
Choose sources → to get a library of your own and a daily brief.
Map each implemented component to the original paper and explain 2 reduced-model failure modes without claiming full-scale reproduction.
Use calibration and shift measurements to justify a monitoring or abstention threshold, and give an explicit confounding counterexample to causal feature-attribution claims.
Defend a causal claim or refusal using the DAG, numerical estimates and a counterexample showing why predictive model fit cannot establish identification.
