Douwe Kiela

Interactive feedback changes how code LLMs perform and rank, a new study finds. Led by senior author Valerie Chen, a Carnegie Mellon PhD student and recent visiting researcher at CDS, the project also included Courant PhD student Jane Pan, CDS PhD student Jacob Pfau, and CDS Assistant Professor He He. The team simulated developer interactions to replace static coding benchmarks like HumanEval and MBPP. When models received human-style feedback—code edits, single-sentence prompts, or paragraph-length instructions—they sometimes improved by up to 9.3 percentage points in accuracy and moved as many as four spots in ranking. “Testing on new problems the models had never seen made feedback even more powerful,” said Chen. Read more: https://lnkd.in/eAPTP5NR

5

Children outperform advanced language models at causal reasoning, especially when explanations are conjunctive rather than disjunctive. CDS PhD student Anthony Chen, working with, among others, Rob Fergus, CDS-associated Professor and Director of AI Research and Head of FAIR at Meta, shows that models like GPT‑4 consistently favor simpler “OR” explanations and falter on “AND” scenarios. The team's inference‑time hypothesis sampling technique, where models generate and test multiple causal hypotheses via Python functions, largely eliminates this bias. https://lnkd.in/d7fQpnej

19

We’re excited to release our latest paper, “Sleep-time Agents: Beyond Scaling Inference at Test-Time”, a collaboration with Charlie Snell from UC Berkeley and Letta advisors and UC Berkeley Faculty Joseph Gonzalez Ion Stoica. We believe there's a fundamental paradigm shift waiting to be unlocked: AI systems that don't just think reactively when prompted, but proactively deepen their understanding during what we call "sleep-time"—the vast periods when they're not directly engaged with users. This is the fundamental insight behind sleep-time compute, a new paradigm we're introducing in our latest research paper. Sleep-time compute represents an exciting new scaling direction for stateful AI systems: by activating deep thinking during their vast idle periods, we can expand what these systems understand and how they reason - pushing AI capabilities beyond what's possible when computation is confined only to active interactions. Letta now allows developers to create sleep-time enabled agents, which have a sleep-time agent processing the message history of the main agent to asynchronously refine that main agent’s in-context memory. Sleep-time agents can also process files or other data to generate *learned context* for the main agent to be more prepared and personalized for future interactions. See details in our paper release: https://lnkd.in/gvz_3Gig If you’re interested in open research on agents, reasoning, memory, and scaling sleep/test-time compute - come join our team - are are actively hiring for research scientists! https://lnkd.in/gghg8SQg

91

2 Comments

A look back at day one of The Hinton Lectures™ featuring Owain Evans, Ph.D., Founder and Director of Truthful AI and researcher at UC Berkeley’s Center for Human-Compatible AI, was a powerful reminder of why Responsible AI matters. Evans spoke about emergent misalignment, how advanced AI models can behave in ways we never intended, and introduced the idea of “subliminal learning,” where models quietly inherit hidden capabilities from one another. His message was clear: We are advancing capability faster than comprehension. That is exactly why the Hinton Lectures exist: to elevate global dialogue on AI safety, transparency, and governance before innovation outruns oversight. At kama.ai, we could not agree more! Our Hybrid AI Agents are built with knowledge graphs, human-in-the-loop governance, and trusted collections because progress only matters if it is aligned, auditable, and accountable. #ResponsibleAI #HybridAI #AIAlignment #HintonLectures #TrustworthyAI #HumanInTheLoop #kamaai

14

1 Comment

🧠 Can AI help detect mental illness earlier? Columbia researchers think so. A new project uses machine learning to spot early signs of schizophrenia in Medicaid data—pushing psychiatry toward faster, more accurate, and more equitable diagnosis. Shalmali Joshi, PhD, a key contributor to this work, is also a proud member of the AI at VP&S Initiative, where experts are harnessing AI to transform health care. 🔗 Read more: https://lnkd.in/e7RgPkbt

7

don't miss out this unique opportunity to join forces on benchmarking time series foundation models on different downstream validation tasks: classification, prediction, anomaly detection. A community effort led by Thomas Moreau from the #MIND team (Centre Inria de Saclay, Inria & CEA Joliot). Still a pending question: Do these model perform well for online real-time forecasting such as in BCI applications, like our play field in BrainSync. Alexandre Gramfort Pedro L. C. Rodrigues Théo Gnassounou Mathieu Dagréou Lucas Struber Jean Faber Joseph Paillard Merlin Dumeur Mathurin Massias

24