ubc-provenance/PIDSMaker: A framework for building provenance-based intrusion detection systems with neural networks (KDD'26, USENIX Sec'25)

📄 Experiments Paper  |  📄 Framework Paper  |  🎙️ NDSS PRISM'26 Slides  |  📘 Documentation  |  ⚙️ Installation

The first framework designed to build and experiment with provenance-based intrusion detection systems (PIDSs) using deep learning architectures. It provides a single codebase to run most recent state-of-the-arts systems and easily customize them to develop new variants.

Release notes

| Version | Date | Highlights | |---------|------|------------| | 3.0.0 | Oct 2026 | Pretrained encoders (SPIDER, CyberGFM, language models, self-supervised GNNs) as node featurization for any PIDS, stable_optim to reduce instability, upgrade many packages including torch 1.13.1 (CUDA 11.7) => 2.1.2 (CUDA 12.1) | | 2.2.0 | Jul 2026 | Interactive 3D embedding viewer, ThreaTrace ground truth | | 2.1.1 | May 2026 | State of the repo for the KDD'26 paper, improved Velox | | 2.1.0 | Apr 2026 | Carbanak v2 and Atlas v2 datasets, dataset download script, Docker and W&B fixes | | 2.0.0 | Jan 2026 | FIVEDIRECTIONS and TRACE datasets, tuned hyperparameters, simplified pipeline stages (renamed arguments) | | 1.0.1 | Oct 2025 | Deterministic graph construction and word2vec, REAPr labels, Apptainer installation, dataset preprocessing scripts | | 1.0.0 | Jun 2025 | Initial release, with 8 PIDSs and the DARPA TC and OpTC datasets |

See the releases for the full notes.

Supported Systems

The framework currently integrates the following PIDSs.

| PIDS | Venue | Paper | |------------|---------------------|-------| | Velox | USENIX Security 2025 | Link | | Orthrus | USENIX Security 2025 | Link | | R-Caid | IEEE S&P 2024 | Link | | Flash | IEEE S&P 2024 | Link | | Kairos | IEEE S&P 2024 | Link | | Magic | USENIX Security 2024 | Link | | NodLink | NDSS 2024 | Link | | ThreaTrace | IEEE TIFS 2022 | Link | | OCR-APT | ACM CCS 2025 | Link |

Supported Datasets

It also includes several easy-to-install provenance datasets for APT detection.

| Dataset | OS | Attacks | Size (GB) | |---------|------|---------|-----------| | CADETS_E3 | FreeBSD | 3 | 10 | | THEIA_E3 | Linux | 2 | 12 | | CLEARSCOPE_E3 | Android | 1 | 4.8 | | FIVEDIRECTIONS_E3 | Windows | 2 | 22 | | TRACE_E3 | Linux | 3 | 100 | | CADETS_E5 | FreeBSD | 2 | 276 | | THEIA_E5 | Linux | 1 | 36 | | CLEARSCOPE_E5 | Android | 2 | 49 | | FIVEDIRECTIONS_E5 | Windows | 4 | 280 | | TRACE_E5 | Linux | 1 | 710 | | optc_h201 | Windows | 1 | 9 | | optc_h501 | Windows | 1 | 6.7 | | optc_h051 | Windows | 1 | 7.7 | | ATLASV2_EDR | Windows | 10 | 1 | | CARBANAKV2_EDR | Windows + Linux | 1 | 6.6 |

Pretrained Encoders

Any PIDS can replace its node featurization (e.g. word2vec) by an encoder pretrained on provenance data.

| Encoder | Description | |---------|-------------| | SPIDER (NeurIPS 2026) | transformer entity encoder distilled from a GNN teacher, with pretrained weights | | CyberGFM | BERT pretrained on random walks, then fine-tuned as a detector | | GPT-2, Llama 3.2, OPT | General-purpose language models fine-tuned on entity labels | | BERT, RoBERTa, ModernBERT, LogBERT, Llama | Language models trained from scratch on random walks | | GraphMAE, GAE, DGI, DeepWalk, node2vec | Self-supervised GNNs and walk embeddings |

See the documentation to download the weights, pretrain an encoder and fine-tune it as a detector.

📄 Documentation

A comprehensive documentation is available, explaining all possible arguments and providing examples on how integrating new systems.

Pipeline

The framework integrates a pipeline composed of seven stages, each parameterizable via configurable arguments, enabling flexible customization of new systems.

Setup

⬇️ Clone the repo

git clone https://github.com/ubc-provenance/PIDSMaker.git

💻 Installation with Docker

We have made the installation of PIDSMaker inclusing pre-processed databases for DARPA TC and OpTC datasets easy and fast. Simply follow these guidelines.

🧪 Basic usage of the framework

Once you have a followed the installation guidelines, you can open a shell in the pids container and experiment in multiple ways. Replace SYSTEM by velox, orthrus, nodlink, threatrace, kairos, rcaid, flash, magic, ocrapt.

1. Run in the shell:

python pidsmaker/main.py SYSTEM DATASET

1. Run in the shell, monitored to weights & biases (W&B):

python pidsmaker/main.py SYSTEM DATASET --wandb

1. Run in background, monitored to W&B (recommended for multiple parallel runs and for research):

./run.sh SYSTEM DATASET

You can still watch the logs in your shell using tail -f nohup.out.

To run in the foreground without W&B, with the same settings as ./run.sh, use ./run_local.sh SYSTEM DATASET.

We generally using using W&B for experiment monitoring and historization (see installation guidelines).

Warning: Before performing evaluations, you should tune all systems (see docs here).

Reproducing results

PIDSs exhibit significant instability—that is, high sensitivity to training perturbations—due to their self-supervised training nature. Running the same configuration with different random seeds or minor hyperparameter changes often yields substantially different results. Consequently, reproducing results as the framework evolves presents a real challenge.

Based on our experiments, we provide tuned hyperparameters for the main systems. However, we can't guarantee that these hyperparameters will lead to satisfactory results due to instability.

We recommend running each system multiple times to increase the likelihood of obtaining a run with good metrics. Alternatively, you can perform hyperparameter tuning for each system.

--training.stable_optim=True (AdamW, warmup and cosine learning rate schedule, gradient clipping) is designed to reduce this instability.

Customize existing systems

The default configuration files in config/*.yml represent the architecture of existing PIDSs in YAML format. They contain the original hyperparameters used by each system.

The main strength of PIDSMaker is the customization of existing systems for easy experimentation. A few examples below.

From CLI

Running Kairos with embedding size of 128 instead of 100, and last neighbor sampling set to last 10 neighbors instead of 20.

python pidsmaker/main.py kairos CADETS_E3 \
  --training.node_hid_dim=128 \
  --batching.intra_graph_batching.tgn_last_neighbor.tgn_neighbor_size=10

Running Orthrus with Doc2vec instead of word2vec, and 3 GraphSAGE layers instead of 2 attention layers.

python pidsmaker/main.py orthrus CADETS_E3 \
  --featurization.used_method=doc2vec \
  --featurization.emb_dim=128 \
  --training.encoder.used_methods=tgn,sage \
  --training.encoder.sage.num_layers=3

From a new YAML config file

Want to create a new PIDS? Create a new config under config/your_system.yml, inherit from existing PIDSs and tune it as you want.

Magic with node type prediction instead of its hybrid masked feature reconstruction and structure prediction objective function, and use a 2-layer MLP with ReLU as decoder, and use NodLink's thresholding method.

`` yaml _include_yml: magic

### Interactive embedding viewer

PIDSMaker also ships an interactive 3D web viewer for exploring a run's node
embeddings — temporal playback, per-node inspection, attack-graph overlays, and
switching between the featurization and GNN-encoder spaces. Train a run with
--save_for_viz to persist the artifacts it needs, then launch it (see the
web viewer README):

!PIDSMaker web embedding viewer

Hyperparameter tuning
PIDSMaker supports easy hyperparameter tuning for existing or new models. 
Follow the instructions available in our documentation.
You can specify the range of hyperparameters to search in a yaml config.

Then run the framework in tuning mode.

Once you find the best hyperparameters, store them in a yaml file and run your tuned model.

## Citation

If you use this work, please cite the two following papers:

Contributing

Pull requests are welcome! Please follow the contribution guidelines.

License

See licence.