10 Critical Insights into Diagnosing Agent Failures in Multi-Agent Systems

Multi-agent systems powered by large language models (LLMs) are transforming how we solve complex problems—but they’re far from perfect. When these systems fail, developers face a maddening puzzle: which agent dropped the ball, and at what moment did things go wrong? Traditional debugging means manually sifting through endless logs, a process as tedious as it is slow. Now, researchers from Penn State University and Duke University, in collaboration with Google DeepMind and other institutions, have introduced a game-changing solution: automated failure attribution. Their work, accepted as a Spotlight presentation at ICML 2025, provides the first benchmark dataset (Who&When) and evaluates several attribution methods. Here are 10 essential things you need to know about this breakthrough.

1. The Growing Complexity of Multi-Agent Systems

LLM-driven multi-agent systems are designed to tackle tasks that single agents can't handle—like collaborative coding, complex reasoning, or multi-step planning. But with more agents comes more moving parts. Each agent may have its own role, knowledge base, and communication protocol. When a task fails, the chain of interactions can stretch over dozens or even hundreds of steps. This complexity turns debugging into a forensic investigation, where the root cause might be a single misinterpretation, a missed message, or a cascading error that only emerges late in the process.

2. The 'Needle in a Haystack' Challenge

Currently, developers debug multi-agent failures by manually reviewing interaction logs—a practice researchers call 'manual log archaeology.' It's time-consuming and error-prone. Suppose a system of five agents runs for 200 steps and fails. Finding the exact agent and step responsible could take hours, even for an experienced engineer. The problem is compounded by the autonomous nature of agents: they make decisions independently, so understanding why a decision led to failure requires deep familiarity with each agent's logic. This bottleneck severely slows down system iteration and optimization.

3. Introducing Automated Failure Attribution

To solve this, the research team formally defined a new problem: automated failure attribution. The goal is to automatically pinpoint which agent caused a failure and at which step of the interaction it occurred. This isn't just about flagging errors—it's about localizing the source with precision. The team argues that without such automation, improving the reliability of multi-agent systems remains a manual, unscalable task. Their work opens a new research direction that could make debugging as simple as running a diagnostic script.

4. The Who&When Benchmark Dataset

The researchers constructed the first benchmark specifically for this task, called Who&When. The dataset contains over 1,000 failure cases from multi-agent systems working on diverse tasks, including question answering, code generation, and decision-making. For each case, the dataset provides the full interaction log, the ground-truth failing agent, and the step where the failure originated. This allows researchers to train and test automated attribution methods fairly. The dataset is publicly available on Hugging Face.

5. How the Benchmark Was Constructed

Building Who&When was a meticulous process. The team simulated multi-agent interactions using popular frameworks like AutoGen and CrewAI. They carefully injected controlled failures at known agents and steps to create ground-truth labels. For example, they might seed an agent with a biased instruction or introduce a communication delay. They also collected real failures from agent logs. Each case was manually verified by multiple annotators to ensure accuracy. The result is a high-quality dataset that captures both synthetic and natural failure scenarios.

6. Two Core Attribution Tasks: Agent and Time

Automated failure attribution breaks down into two sub-problems: agent attribution (which agent caused the failure?) and time attribution (at which step did the failure originate?). The researchers evaluate methods on both dimensions separately and jointly. They find that these tasks are complementary—knowing the failing agent helps narrow down the time window, and vice versa. A system that solves both simultaneously offers the most practical value for developers who need actionable insights quickly.

7. Baseline Methods Evaluated

The paper tests several automated attribution methods, ranging from simple heuristics to advanced machine learning models. Heuristics include: selecting the agent that generated an invalid output, or the agent at the step where the last correct action occurred. They also evaluate supervised learning models trained on log features, and even large language model-based reasoners that analyze the full log. The results show that no single method excels across all scenarios, highlighting the difficulty of the task and the need for more research.

8. Key Findings from the Research

One surprising finding is that LLM-based reasoning methods perform reasonably well on short interaction logs but degrade quickly as the number of steps increases. In contrast, simpler heuristics can be surprisingly effective for specific failure types, like clear invalid outputs. The best overall performance comes from a hybrid approach: first use a classifier to identify the likely failing agent, then use an LLM to verify and refine the time step. Even so, the top method still leaves room for improvement, with accuracy around 70% on the hardest cases.

9. Implications for Developers and Researchers

For developers, this work offers a tangible starting point for building debugging tools into multi-agent systems. Integrating automated attribution could slash the time spent on log analysis from hours to minutes. For researchers, the Who&When benchmark provides a standardized testbed for comparing future methods. The code and dataset are fully open-source (GitHub), enabling the community to build on this foundation and drive progress in system reliability.

10. Future Directions and Open Problems

The study also identifies several open challenges. First, attribution methods need to scale to systems with dozens of agents and thousands of steps. Second, they must handle ambiguous failures where multiple agents contribute. Third, real-time attribution—flagging failures as they happen—remains an unsolved problem. The researchers call for more work on proactive failure prediction, and for integrating attribution with automated repair. As multi-agent systems become more common, solving these issues will be critical for building trustworthy AI.

Automated failure attribution is still in its infancy, but this research lays a solid foundation. By answering the twin questions of 'who?' and 'when?', it empowers developers to iterate faster and create more resilient LLM multi-agent systems. The Who&When dataset and open-source tools are now available—so dive in, test your own methods, and help push the field forward.