Recommendation · MoneyPeople don't really test their memory systems — test yours. Ramp's 'roll forward worlds' technique: run the same task over two time periods, write a memory after the first task, then rerun the task in the second period, either injecting the memory directly (cheating a little) or letting the agent search over a set of memories. Design tasks so implicit rules must be inferred from the success criteria, and use a memory prompt that compresses, writes, then reruns and reads the memory back. This produced one of the biggest gains Ramp saw from these types of evals.
AI Engineer · How Ramp Tests Its AI Bookkeeping Agent Before Customers Do - Ryan Stevens, Ramp
Recommendation · MoneySuraj Gupta recommends using persistent memory so future runs of an agent can lean on what has already been done in the past, and warns that without it, agents encountering similar issues repeatedly waste a lot of tokens regathering context on issues already fixed before — even if the agent figures out the same root cause again.
hedged
AI Engineer · Building Self-Improving Agent Software Factories — Suraj Gupta, Warp
Recommendation · MoneySuraj Gupta recommends choosing which models to use for which tasks: define classes of tasks in a routing configuration file and assign each class to a specific model, or as a new user use Warp's out-of-the-box auto model routers, which evaluate the best models for Pareto efficiency as new models come out. He discourages pinning your model to Opus or Haiku as a new user, saying auto models are much better.
AI Engineer · Building Self-Improving Agent Software Factories — Suraj Gupta, Warp
Recommendation · MoneySuraj Gupta recommends that when building a self-improving software factory, you focus on three concrete ways agents can improve: skills, persistent memory, and model routing.
AI Engineer · Building Self-Improving Agent Software Factories — Suraj Gupta, Warp
Method · MoneyWarp's internal software factory runs a skill self-improvement loop: (1) a workflow runs a triage agent every time an issue is submitted on the repo, with a skill describing how to reproduce issues and figure out what information the user did not provide (something to build, an existing issue to dedupe against, or something else); (2) an outer loop agent looks at the decisions the inner loop triage agent made and collects feedback signals such as thumbs up/down on issues, user comments, or Warp employee comments; (3) the feedback signals are synthesized into an iteration that updates the inner loop skill, which is what leads to the self-improvement loop with skills; (4) the outer loop agent edits the triage skill and opens a pull request, so all improvements to the inner loop skill are Git
hedged
Method, not a verified protocol.
AI Engineer · Building Self-Improving Agent Software Factories — Suraj Gupta, Warp
Recommendation · MoneyThe creator describes and endorses using a mixture of pellets, usually soy hull and hardwood, as the substrate for growing mushrooms commercially, noting availability depends on what can be sourced in your area. They frame this pellet method as probably the most standard and most commercial way to grow mushrooms, so starting small allows you to scale big.
The creator hedges that this is 'probably' the most standard and most commercial way to grow mushrooms.
Mountain View Mushrooms · Starting A Small Scale Mushroom Farm
Recommendation · MoneyThe creator emphasizes having a humidifier with a tank in your grow room, because without it you can't keep the mushrooms moist and they can't get nice and big and plump.
Mountain View Mushrooms · Starting A Small Scale Mushroom Farm
Recommendation · MoneyThe creator recommends sterilizing substrates using a Presto pressure canner as the simplest and fairly cheap way, heated with an induction heater or an outdoor propane burner. Load the cooker with as many bags as possible, with five pound blocks being ideal, and cook until the substrates are sterile. The same method is used for grains so the mycelium has something nutritious to eat.
Mountain View Mushrooms · Starting A Small Scale Mushroom Farm
Recommendation · MoneyThe creator advises using exhaust fans to pull and push air through the plenum room, grow room, and outside, because without them mushrooms won't get the oxygen they need to look nice and pretty and CO2 concentration in the room will be too high.
Mountain View Mushrooms · Starting A Small Scale Mushroom Farm
Recommendation · MoneyThe creator advises getting a grow room to actually grow the mushrooms once the blocks on the incubation shelf are done incubating, and having some way to control the temperature in the grow room, such as a mini split unit.
hedged
Mountain View Mushrooms · Starting A Small Scale Mushroom Farm
Recommendation · FuelThe host's stated recommendation: if you're buying mushroom coffees for functional benefits, read up on the doses required to achieve that effect from proper human randomized control trials.
hedged
Also discussed by Eightbillion (unclassified source — not counted until classified). Not counted as independent sources.
James Hoffmann · What's The Deal With Mushroom Coffee?
Recommendation · MoneyKanazawa advises iterating on the rubric creation process itself rather than expecting it right the first time: direct the agent on how rubrics should be created instead of writing them yourself, and expect to iterate quite a bit before they work well. When reviewing generated output, identify specific elements you don't like and tell the agent to change the rubric in a particular way so the output matches your intent.
AI Engineer · I Let Agents Refactor My Codebase for 3 Weeks. Then I Read the Code. — Keiji Kanazawa, Microsoft
Recommendation · MoneyKanazawa has Claude and Codex check each other's work in adversarial code reviews where they deeply look at what they're doing and suggest fixes, running for several rounds. He also had Codex and Claude take turns implementing, reviewing each other's code, and fixing each other's bugs.
AI Engineer · I Let Agents Refactor My Codebase for 3 Weeks. Then I Read the Code. — Keiji Kanazawa, Microsoft
Recommendation · MoneyWhen a coding agent produces a wrong output, Kanazawa advises doing error analysis: look at traces of what the agent did and write evals so it improves in the future. He asked Claude Code to tell him for a particular session where the traces are.
hedged
AI Engineer · I Let Agents Refactor My Codebase for 3 Weeks. Then I Read the Code. — Keiji Kanazawa, Microsoft
Recommendation · MoneyKanazawa warns that agents will cheat by creating rubrics tuned to succeed on only one specific input (bench-maxxing). Failing to review the rubrics completely allows this: a rubric over-tuned to one video fails when run on other videos. Telling the agent not to game itself to be good at one video makes it generalize better and score higher overall.
AI Engineer · I Let Agents Refactor My Codebase for 3 Weeks. Then I Read the Code. — Keiji Kanazawa, Microsoft
Recommendation · MoneyKanazawa advises setting a rubric specifying desired output qualities before letting agents generate content. For a video canvas he specifies: everything well explained, everything simple, and the ability to jump to the right places in the video.
AI Engineer · I Let Agents Refactor My Codebase for 3 Weeks. Then I Read the Code. — Keiji Kanazawa, Microsoft
Recommendation · MoneyAyush endorses one way to ship several of these at once: an agent mode, where your CLI notices an agent is calling it and prints differently.
AI Engineer · Your Agent Hates Your CLI - Ayush Bhardwaj, Allos AI
Recommendation · MoneyAyush's advice: fail loudly. When a tool fails, make the very last lines a short summary and a path to the full log.
AI Engineer · Your Agent Hates Your CLI - Ayush Bhardwaj, Allos AI
Recommendation · MoneyAyush advises that you can get real levels and test cases for your tool easily: if you use it every day, they are already in your transcripts. Replay those commands through your old build and your new one.
AI Engineer · Your Agent Hates Your CLI - Ayush Bhardwaj, Allos AI
Recommendation · MoneyThe source lays out scaling by budget: start a mushroom business with a $20 grow kit from a grocery store to grow your first batch (you can sell those mushrooms if you want). Once past the grow-kit phase, you can expand onto substrate from a local grocery store for around $150 total, needing a pot. You might get away with a ~$50 build by taking the $20 grow kit, breaking it apart, and putting it on substrate material where it will actually grow; mixing a $20 grow kit with $30-40 of substrate can grow 5, 10, or 20 times as much mushroom as the kit alone. With a $1,000 budget, buy a grow tent first so the mushrooms have a good environment to grow in — that's the biggest next step. With a $10,000 budget, get a whole shed and convert it: a small lab area in one section and a grow room with an
might, probably
UpFlip · Here's How to Start a $10K/Month Mushroom Business from Home