tests/test.py . [100%]

============================== 1 passed in 7.16s ===============================

## How-to-use

Data format
The telemetry data must be presented as pandas.DataFrame. We require the data to have a column named time that stores the timestep. A sample of valid data could be downloaded using the download_data() or download_multi_source_data() method that we will demonstrate shortly below.

Converting your own telemetry
With pip install RCAEval[convert], rcaeval-convert builds a case directory in the format above from your own system: metrics from Prometheus, traces from OpenTelemetry (OTLP/JSON, which Jaeger, Zipkin and SkyWalking can export through the OpenTelemetry Collector), and logs from a JSON Lines, JSON or CSV export of any log store (ELK, Loki, Fluentd, ...). It also derives the log and trace time series the multi-source methods read. See docs/CONVERT.md.

### Basic usage example

A basic example to use BARO, a metric-based RCA baseline, to perform RCA are presented as follows,

You can put the code here to a file named test.py

download a sample data to data.csv

read data from data.csv

perform root cause analysis

print the top 5 root causes

Expected output after running the above code (it takes around 1 minute)

A tutorial of using Multi-source BARO to diagnose failure using multi-source telemetry data (metrics, logs, and traces) is presented in docs/multi-source-rca-demo.ipynb.
A tutorial of using BARO to diagnose code-level faults is presented in docs/code-level-rca.ipynb.
Evaluating LLM agents
To evaluate an LLM agent, or any RCA method that does not run inside main.py, export a dataset as a blind task view, run your agent over it, and score the predictions file it writes:

The task view renames every case to an anonymised id (the raw case directories are named after their root cause) and keeps the labels in a separate answers file. Predictions are scored with the same AC@k and Avg@k as main.py.

With pip install RCAEval[agent], RCAEval also ships the agents themselves. rcaeval-agent run --model <any LiteLLM model> runs a reference LLM agent over the task view. rcaeval-agent run-cli --agent claude-code lets an agent CLI investigate each case through RCAEval's MCP server. Both use the same tools for querying metrics, logs and traces, and both report cost and whether the agent looked at the root-cause service. See docs/AGENTS.md.

Available Datasets
RCAEval benchmark includes nine datasets organized into three benchmark suites (RE1, RE2, RE3), each covering three microservice systems (Online Boutique, Sock Shop, Train Ticket). Together, these datasets feature 735 failure cases with 11 fault types. Each failure case includes annotated root cause service and root cause indicator (e.g., specific metric or log indicating the root cause).
| Dataset | System | Cases | Fault Types | Metrics | Logs | Traces |
|---------|--------|-------|-------------|---------|------|--------|
| RE1-OB | Online Boutique | 125 | cpu, mem, disk, delay, loss | 49-59 | N/A | N/A |
| RE1-SS | Sock Shop | 125 | cpu, mem, disk, delay, loss | 57-63 | N/A | N/A |
| RE1-TT | Train Ticket | 125 | cpu, mem, disk, delay, loss | 198-238 | N/A | N/A |
| RE2-OB | Online Boutique | 90 | cpu, mem, disk, delay, loss, socket | 69-77 | Yes | Yes |
| RE2-SS | Sock Shop | 90 | cpu, mem, disk, delay, loss, socket | 74-82 | Yes | N/A |
| RE2-TT | Train Ticket | 90 | cpu, mem, disk, delay, loss, socket | 340-376 | Yes | Yes |
| RE3-OB | Online Boutique | 30 | f1, f2, f3, f4, f5 | 68-101 | Yes | Yes |
| RE3-SS | Sock Shop | 30 | f1, f2, f3, f4 | 80-107 | Yes | N/A |
| RE3-TT | Train Ticket | 30 | f1, f2, f3, f4 | 294-322 | Yes | Yes |
| TORAI-OB | Online Boutique | 90 | cpu, mem, disk, delay, loss, socket | 69-77 | Yes | Yes |
| TORAI-SS | Sock Shop | 90 | cpu, mem, disk, delay, loss, socket | 74-82 | Yes | N/A |
| TORAI-TT | Train Ticket | 90 | cpu, mem, disk, delay, loss, socket | 340-376 | Yes | Yes |
RE1 Datasets (375 cases): Metric-only data supporting metric-based RCA methods. Five fault types (CPU, MEM, DISK, DELAY, LOSS) across five services per system, with five repetitions per fault-service pair.
RE2 Datasets (270 cases): Multi-source data (metrics, logs, traces) supporting multi-source RCA methods. Six fault types (adds SOCKET) across five services per system, with three repetitions per fault-service pair.
RE3 Datasets (90 cases): Multi-source data focusing on code-level faults (F1-F5). Supports diagnosing code-level faults through telemetry data, e.g., leveraging stack traces in logs or response codes in traces.
File Structure
Each dataset directory follows the naming convention: {benchmark}_{service}_{fault}_{instance}
metrics.json: Time-series metrics data

inject_time.txt: Fault injection timestamp (Unix timestamp)

logs.csv: Log data (RE2 and RE3 only)

traces.csv: Trace data (RE2 and RE3 only)

Our datasets and their description are publicly available with the following information:
Hugging Face (recommended): https://huggingface.co/datasets/phamquiluan/RCAEval (Parquet, 3.4GB on disk, downloadable one suite or one case at a time)
Figshare: https://figshare.com/articles/dataset/RCAEval_A_Benchmark_for_Root_Cause_Analysis_of_Microservice_Systems/31048672 (more structured format, with additional preprocessed series)
Zenodo: https://zenodo.org/records/14590730 (DOI for citation: https://doi.org/10.5281/zenodo.14590730)
The Hugging Face copy stores the same 735 cases as Parquet instead of metrics.json /
logs.csv / traces.csv. It is 3.4GB to download against 5.2GB of zip archives on
Zenodo, and about eleven times smaller once extracted, since those archives expand to
roughly 38GB. Unlike a zip archive it can also be fetched a suite or a single case at a
time, so you never download telemetry you will not use:

one suite (use "re2" or "re3" for the others, or omit to get all 735 cases)

cases.parquet in that repository indexes all 735 cases with their ground-truth root
cause service, fault type, injection time, and telemetry sizes, so you can select a
subset before downloading any telemetry:

idx = pd.read_parquet("hf://datasets/phamquiluan/RCAEval/cases.parquet") print(idx[idx.dataset == "RE2-TT"][["case", "root_cause_service", "fault", "n_metrics"]])

RCAEval.utility.read_metrics, read_logs and read_traces read either layout, so code
written against the original files works unchanged with the Parquet copy:

df = read_metrics("data/re1ob_adservice_cpu_1") # metrics.parquet or metrics.json

We also provide utility functions to download our datasets using Python. The downloaded datasets will be available at directory data.

download_re1_dataset() download_re2_dataset() download_re3_dataset()

<details>
<summary>Expected output after running the above code (it takes half an hour to download and extract the datasets. )</summary>

</details>

Available Baselines 
RCAEval stores all the RCA methods in the e2e module (implemented in RCAEval.e2e). There are many RCA baselines available: RUN, CausalRCA, CIRCA, RCD, MicroCause, MSCRED, BARO, 𝜖-Diagnosis, TraceRCA, MicroRank, Multi-source BARO, Multi-source RCD, Multi-source CIRCA, TORAI, EventADL.

Benchmark 
We provide a script named main.py to assist in reproducing the results from our RCAEval paper. This script can be executed using Python with the following syntax:

The available options and their descriptions are as follows:

MicroCause and CloudRanger start their search from the SLI, which main.py sets to the entry service's latency. Their paper numbers were produced by RCAEval 1.11.0 and earlier, which used the root-cause service's latency instead (#88), so this version's numbers for these two methods differ from the paper.

For example, in Table 6, BARO achieves Avg@5 of 0.72, 0.99, 1, 0.83, 0.64, and 0.8 for CPU, MEM, DISK, SOCKET, DELAY, LOSS, and AVERAGE on the Train Ticket dataset. To reproduce these results, you can run the following commands:

The expected output should be exactly as presented in the paper (it takes less than 1 minute to run the code)

We can replace the baro method with other methods (e.g., circa) and substitute re2-tt with other datasets to replicate the corresponding results shown in Table 6. This reproduction process is also integrated into our Continuous Integration (CI) setup. For more details, refer to the .circleci/config.yml file.

Multi-source RCA benchmark
The multi-source methods (Multi-source BARO, Multi-source RCD, and Multi-source CIRCA) consume metrics, logs, and traces together. They run on the RE2/RE3 datasets, which include the required logts.csv, tracets_err.csv, and tracets_lat.csv files in each failure case:

Multi-source BARO (Python 3.14, default environment)

Multi-source CIRCA (Python 3.14, default environment)

A function-level tutorial using Multi-source BARO on a single failure case is available in docs/multi-source-rca-demo.ipynb.
For TORAI Paper
TORAI Datasets (270 cases): A processed version of the RE2 datasets, containing pre-aggregated time-series for metrics, logs, and traces (where available). These datasets are used by the TORAI method and are available separately on Figshare: https://doi.org/10.6084/m9.figshare.31925976.
TORAI requires a separate Python 3.8 environment and its own datasets. For full instructions, see docs/TORAI.md for installation and reproducibility instructions.

<details>
<summary>Expected output (it takes less than 1 minute)</summary>

</details>

For EventADL Paper
EventADL localizes root-cause actors/resources over CloudTrail-style API-call event logs, rather than the metrics/traces/logs used by other RCAEval methods. It runs in the default environment. For full instructions, see docs/EVENTADL.md for installation and usage instructions.

Publications

Anomaly Detection and Root Cause Analysis for Microservice Systems
PhD Thesis, RMIT University, 2026 · Luan Pham

EventADL: Open-Box Anomaly Detection and Localization Framework for Events in Cloud-Based Service Systems
FSE 2026 · Luan Pham, Victor Nicolet, Joey Dodds, Hui Guan, Daniel Kroening

TORAI: Multi-source Root Cause Analysis for Blind Spots in Microservice Service Call Graph
FSE 2026 · Luan Pham, Huong Ha, Xiuzhen Zhang, Hongyu Zhang

Where Root Cause Analysis Fails: A Retrieval-Reranking Decomposition
NeurIPS 2026 (Evaluations and Datasets Track) · Hada Melino Muhammad, Luan Pham, Laure Barrière, Sachin Shetty, Leonardo Pulga, Flora D. Salim

RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data
WWW 2025 (Companion) · Luan Pham, Hongyu Zhang, Huong Ha, Flora Salim, Xiuzhen Zhang

Root Cause Analysis for Microservice System based on Causal Inference: How Far Are We?
ASE 2024 · Luan Pham, Huong Ha, Hongyu Zhang

BARO: Robust Root Cause Analysis for Microservices via Multivariate Bayesian Online Change Point Detection
FSE 2024 · Luan Pham, Huong Ha, Hongyu Zhang

Licensing

This repository includes code from various sources with different licenses. We have included their corresponding LICENSE into the LICENSES directory:

• BARO: Licensed under the MIT License. Original source: BARO GitHub Repository.
• CausalRCA: No License. Original source: CausalRCA GitHub Repository.
• CIRCA: Licensed under the BSD 3-Clause License. Original source: CIRCA GitHub Repository.
• E-Diagnosis: Licensed under the BSD 3-Clause License. Original source: PyRCA GitHub Repository.
• MicroCause: Licensed under the Apache License 2.0. Original source: MicroCause GitHub Repository.
• RCD: Licensed under the MIT License. Original source: RCD GitHub Repository.
• RUN: No License. Original source: RUN GitHub Repository.

Acknowledgments

We would like to express our sincere gratitude to the researchers and developers who created the baselines used in our study. Their work has been instrumental in making this project possible. We deeply appreciate the time, effort, and expertise that have gone into developing and maintaining these resources. This project would not have been feasible without their contributions.

Change Logs