31 results · Most relevant
Kaggle is building standardized infrastructure so that every benchmark score is auditable, high quality, and repeatable. This includes sandboxing every run, logged proxy calls, open traces for every model call, and containerized grading.
Sonoco is a U.S. packaging company that recently bought a company called Eviosis and sold one of its businesses called TFP. To value Sonoco, the first basic number needed is sales, but five different sources give different sales numbers, ranging from $5 billion to $8 billion, because each source counts the difference on OCO differently. The agent has to first understand that the numbers don't match, then find out why, and then rebuild Sonoco as it was in May 2025, with Eviosis for the full year and without TFP, which will be about $7.5 billion of sales. Rebuilding the company's sales figure is just a small part of the entire reasoning a skilled analyst works through.
In preliminary evaluation, every model behaves differently across task types — e.g., one model may excel at financial modeling while another is balanced. Relying on a single benchmark score hides these differences.
moderate — based on a preliminary 10-task run; model names in the transcript are uncertain (possible numeric error)
In preliminary results over 10 of the 50-task universe, Claude with 5.5 scores best. Every model behaves differently on the benchmark; for example, Astrus 6 is really good at financial modeling, while Office 5.5 is a balanced model that performs very well across different tasks.
preliminary results only cover 10 of 50 tasks
Kaggle has spent 15 years evaluating humans on real-world capabilities, built a community of 31 million AI and ML enthusiasts, and run thousands of competitions. Its success is attributed to rigor: sealed private test datasets, leak detection, and anti-cheating infrastructure. In February 2026, Kaggle moved into Google DeepMind and shifted from evaluating humans to evaluating frontier models.
The full 50-task MandateBench benchmark will be launching next month in November 2026, and it is part of many other benchmarks launching on Kaggle and DeepMind over the next few weeks. Google DeepMind has partnered with leading AI startups to build benchmarks in different domains: Hey Jen's Code to Video Bench launched the week before this talk, and GDM is working on benchmarks with Octava, Quadrillion, and Rehearsals on clinician workflows, research workflows, and user research workflows.
Google DeepMind partnered with leading AI startups to build benchmarks in different domains, incentivizing participation with free model compute, FTE support, and $5 million in grants.
LinqAlpha builds AI research agents for institutional investors; institutions with more than $5 trillion in assets under management use LinqAlpha, and more than 200 investors use it actively. LinqAlpha is also a research team that has published 13 papers at venues like iClear or ICML, co-authored with BlackRock, Blackstone, and JP Morgan.
No published protocol matches this selection.
The creators behind most of what cleared.
28 cleared insights match
Choose sources → to get a library of your own and a daily brief.
