ONNX Simplifier

import torch
class JustReshape(torch.nn.Module):
    def __init__(self):
        super(JustReshape, self).__init__()
def forward(self, x):
        return x.view((x.shape[0], x.shape[1], x.shape[3], x.shape[2]))
net = JustReshape()
model_name = 'just_reshape.onnx'
dummy_input = torch.randn(2, 3, 4, 5)
torch.onnx.export(net, dummy_input, model_name, input_names=['input'], output_names=['output'])

!simple_reshape

However, I got the following complicated model instead:

!complicated_reshape

Our solution

ONNX Simplifier is presented to simplify the ONNX model. It infers the whole computation graph and then replaces the redundant operators with their constant outputs (a.k.a. constant folding).

Features

At its core onnxsim runs a fixed point of shape inference, graph optimization and constant folding until the model stops changing. Around that it offers:

• Constant folding. Evaluates the constant parts of the graph and replaces

• Graph optimization passes. Runs onnx-optimizer's fusions and eliminations

• Shape inference. Propagates tensor shapes through the graph — including

• Correctness checking. Optionally validates the simplified model against the

• [Custom operators. Keeps custom ops (TensorRT plugins,

• Opset conversion. Upgrade or downgrade the

• Function inlining. Flatten the model's local (model-defined) functions into

• Custom rewriters. Plug your own rewriting logic into

• Safetensors / GGUF archives. Export a model

• Transformers export. Export a Hugging Face

• Diffusion model export. Export a Hugging

• Detectron2 export. Trace a

• SAM 2 export. Trace a Meta

• DriveTransformer export. Trace a

• Deploying to Axelera Metis devices.

• Quantization-aware fine-tuning.

• Subgraph simplification. Simplify If/Loop/Scan subgraph bodies too

• MLIR export. Hand the simplified

• Core ML export. Convert the simplified model to a

• TensorFlow Lite export. Convert the

• Large-model handling. Guard against blow-up from ops like Tile/

• Many ways to run it. A zero-install web version, a Python

Getting started

Web version

We have published ONNX Simplifier on GitHub pages. It works out of the box and doesn't need any installation. Note that it runs in the browser locally and your model is completely safe.

Python version

pip3 install -U pip && pip3 install onnxsim

Then

onnxsim input_onnx_model output_onnx_model

For more advanced features, try the following command for help message

onnxsim -h

onnx is the only required dependency. Everything else is an extra, installed only if you want it -- among them rich, which is used purely to colour and box the terminal reports (the original-vs-simplified table, the memory plan, the graph diff, CLI warnings). Without it those print as plain-text ASCII tables and the simplified models are byte-for-byte the same:

pip3 install "onnxsim[rich]"

Node.js version

The same WebAssembly build backing the web version above is also published as an npm package, for JavaScript tooling that wants ONNX simplification without a native build step or a Python runtime. See npm/onnxsim/README.md for usage.

npm install onnxsim

Demonstration

An overall comparison between a complicated model and its simplified version:

!Comparison between old model and new model

In-script workflow

If you would like to embed ONNX simplifier python package in another script, it is just that simple.

import onnx
from onnxsim import simplify
load your predefined ONNX model
model = onnx.load(filename)
convert model
model_simp, check = simplify(model)
assert check, "Simplified ONNX model could not be validated"
use model_simp as a standard ONNX model object

You can see more details of the API in onnxsim/onnx_simplifier.py

Custom operators

Models that contain custom operators, such as TensorRT plugins (BatchedNMS_TRT, EfficientNMS_TRT, ...), are supported. onnxsim keeps these ops unchanged and simplifies the rest of the graph around them. This works whether the custom op lives in a vendor-specific domain (e.g. TRT) or in the default ONNX domain, so you no longer need to manually move it into a custom domain to get past validation (issues #107 and #220).

onnxsim also ships schemas out of the box for a few specific custom-op families, so shape inference propagates through them with no setup at all: ONNX Runtime's com.microsoft quantized/contrib ops, mmdeploy/mmcv/BEVDet's custom ops, and -- see docs/qonnx-brevitas-interop.md -- Brevitas's native QONNX export format (Quant/BipolarQuant/Trunc/FloatQuant, in the qonnx.custom_op.general or finn.custom_op.general domain). A Brevitas QAT export's learned quantizers are also picked up by onnxsim.qat_interop's ingest path (quantize_static_keeping_qdq_scales), the same as a QDQ-exported QAT model's -- see that doc and the "Quantization-aware fine-tuning" section below.

If you describe your custom operator to ONNX with onnx.defs.register_schema, onnxsim picks that schema up automatically: onnxsim links its own copy of ONNX, so its operator registry is separate from the onnx Python module's, and every simplify call imports the schemas you registered into onnxsim's registry before validating the model (issue #326). You can also trigger the import explicitly with onnxsim.import_onnx_schemas(), or turn the automatic import off with onnxsim.simplify(model, import_custom_schemas=False) (CLI: --skip-schema-import).

import onnx
import onnxsim
Teach ONNX about your custom operator.
onnx.defs.register_schema(my_op_schema)
simplify() imports the schema into onnxsim automatically.
model_simp, check_ok = onnxsim.simplify(model)

If a registered schema also has a type/shape-inference function (set via onnx.defs.OpSchema.set_type_and_shape_inference_function), onnxsim registers a trampoline that calls it back through onnx.shape_inference.infer_node_outputs during simplification, so the custom operator's output shapes are inferred too. Custom operators without an inference function are still imported; shape inference simply flows past them.

Changing the opset version

You can upgrade (or downgrade) the model's opset version while simplifying. Pass target_opset_version to simplify (CLI: --target-opset) and onnxsim converts the default ONNX domain to that opset — using onnx's own version converter — before running the simplification, so any redundant nodes the conversion introduces get cleaned up too.

import onnx
import onnxsim
model = onnx.load(filename)
Convert the model to opset 18 and simplify it.
model_simp, check = onnxsim.simplify(model, target_opset_version=18)

On the command line:

onnxsim input_onnx_model output_onnx_model --target-opset 18

When target_opset_version is left unset (the default), the model's opset version is preserved.

The conversion runs inside onnxsim's C++ core, so every binding shares it — the Python package, the C API and its Rust wrapper (Options::target_opset_version), the standalone onnxsim binary (--target-opset), and the web version (the "target opset version" field).

Exporting to MLIR (torch-mlir / onnx-mlir)

Downstream compiler stacks built on MLIR — torch-mlir, IREE on top of it, and onnx-mlir — consume models as MLIR rather than as an ONNX ModelProto. onnxsim can bridge the gap: after simplifying, it emits the model as MLIR in one of two dialects, chosen with --mlir-target (Python: the target argument):

• torch (default) — Torch-dialect MLIR via torch-mlir's pure-Python

• onnx — ONNX-dialect MLIR via the onnx-mlir compiler binary.

Both backends are optional (just like onnxruntime for constant folding): neither is imported/located unless you actually emit MLIR.

torch-mlir (Torch dialect)

Install torch-mlir:

pip install torch-mlir

Prebuilt wheels are listed at .

From the CLI, add --emit-mlir. Passed without a path it writes the MLIR next to the output model with a .mlir extension; pass a path to choose the location:

# writes simplified.onnx and simplified.mlir
onnxsim input.onnx simplified.onnx --emit-mlir
choose the MLIR path explicitly
onnxsim input.onnx simplified.onnx --emit-mlir model.mlir

From Python, onnxsim.export_mlir converts a model (typically the output of simplify) and returns the MLIR text, optionally writing it to a file:

import onnx
import onnxsim
model = onnx.load("input.onnx")
model_simp, ok = onnxsim.simplify(model)
assert ok
Return the MLIR as a string...
mlir_text = onnxsim.export_mlir(model_simp)
...and/or write it to a file.
onnxsim.export_mlir(model_simp, "model.mlir")

onnx-mlir (ONNX dialect)

onnx-mlir has no pip-installable importer, so this backend shells out to the onnx-mlir compiler binary (--EmitONNXIR). Build or install it from , then make it discoverable — put onnx-mlir on your PATH, set ONNX_MLIR_HOME to its install prefix (the binary is expected at $ONNX_MLIR_HOME/bin/onnx-mlir), set ONNX_MLIR to the binary path, or pass the path explicitly.

# locate onnx-mlir via PATH / ONNX_MLIR_HOME / ONNX_MLIR
onnxsim input.onnx simplified.onnx --emit-mlir --mlir-target onnx
or point at the binary directly
onnxsim input.onnx simplified.onnx --emit-mlir model.mlir \
  --mlir-target onnx --onnx-mlir /path/to/onnx-mlir

mlir_text = onnxsim.export_mlir(model_simp, target="onnx")
with an explicit binary path:
onnxsim.export_mlir(model_simp, "model.mlir", target="onnx",
                    onnx_mlir="/path/to/onnx-mlir")

export_mlir accepts a few keyword arguments, forwarded to the selected backend — e.g. opset_version to run ONNX's version converter first (both targets prefer recent opsets), verify=False (torch) to skip MLIR verification, and emit / extra_args (onnx) to change the onnx-mlir emit flag or pass extra compiler options. See onnxsim/mlir_export.py for the full signatures.

Exporting to Core ML

Apple platforms want the graph as a Core ML model instead of ONNX or MLIR. coremltools dropped its own ONNX frontend in version 7 (it only converts TensorFlow/PyTorch models, or an in-memory MIL program) — there's no off-the-shelf "convert this ONNX model" call left to lean on, so onnxsim ships its own ONNX-to-MIL translator and hands the result to coremltools' MIL-to-Core-ML backend to produce the actual model. It covers a practical subset of ONNX ops (conv/pooling/normalization, matmul/gemm, elementwise math, reshapes, reductions, and more — see coreml_export.SUPPORTED_ONNX_OPS); a node whose op isn't supported raises a clear error naming the op, rather than silently producing a wrong model. Feeding in a simplified model is the point, same as with MLIR export: onnxsim's constant folding turns more of the graph into plain initializers, so more of it lands on the translator's supported-op list.

coremltools is optional, just like onnxruntime for constant folding: it isn't imported unless you actually export to Core ML.

pip install coremltools

Converting an ONNX model to MIL / Core ML needs no macOS-specific functionality (MIL construction and .mlpackage serialization are pure Python/protobuf), so it runs the same on Linux, macOS, or Windows. Only loading the produced model back for a prediction needs Core ML's runtime, i.e. an Apple OS — pass skip_model_load=False (Python) once you're on macOS to get a model that's ready to call .predict() on; the default (skip_model_load=True) lets conversion succeed everywhere else too.

Graph inputs must have fully static shapes (dynamic axes aren't supported).

From the CLI, add --emit-coreml. Passed without a path it writes the model next to the output model with a .mlpackage/.mlmodel extension; pass a path to choose the location:

# writes simplified.onnx and simplified.mlpackage
onnxsim input.onnx simplified.onnx --emit-coreml
choose the path and the legacy .mlmodel format explicitly
onnxsim input.onnx simplified.onnx --emit-coreml model.mlmodel --coreml-format neuralnetwork

From Python, onnxsim.export_coreml converts a model (typically the output of simplify) and returns the coremltools.models.MLModel, optionally saving it:

import onnx
import onnxsim
model = onnx.load("input.onnx")
model_simp, ok = onnxsim.simplify(model)
assert ok
Return the MLModel...
mlmodel = onnxsim.export_coreml(model_simp)
...and/or save it to a .mlpackage (or .mlmodel with convert_to="neuralnetwork").
onnxsim.export_coreml(model_simp, "model.mlpackage")

export_coreml accepts a few keyword arguments: convert_to ("mlprogram", the default, or the legacy "neuralnetwork"), compute_units (which devices the model may run on, e.g. "CPU_ONLY"), compute_precision, minimum_deployment_target (e.g. "iOS16"), io_dtype (see below), and skip_model_load (see above). See onnxsim/coreml_export.py for the full signature.

io_dtype="fp16" (CLI: --coreml-io-dtype fp16) declares the model's float inputs and outputs float16 instead of float32. An ML Program already computes in float16, so the float32 default only buys a conversion in each direction on every call, over twice the bytes — with no accuracy difference, since a float32 output is just an upcast of the float16 value Core ML computed either way. It's worth most where the same large float tensors cross the boundary repeatedly, as a transformer decoder's KV cache does on every generated token. Requires convert_to="mlprogram" and raises the deployment target to iOS16/macOS13 when one isn't given. See scripts/apple/README.md's "fp16 model interface" section.

Exporting to TensorFlow Lite

Mobile/embedded runtimes built on TensorFlow want the graph as a .tflite flatbuffer instead of ONNX or Core ML. onnx-tensorflow/onnx-tf (the project that used to fill this gap) has been unmaintained for years and only tracks very old opsets, so -- same situation as Core ML after coremltools dropped its own ONNX frontend -- onnxsim ships its own ONNX-to-TensorFlow translator: it builds the equivalent computation with plain TensorFlow ops inside a tf.function, traces it into a concrete function, and hands that to tf.lite.TFLiteConverter to produce the actual .tflite model. It covers a practical subset of ops (conv/pooling/normalization incl. LayerNormalization, matmul/gemm, elementwise math incl. comparisons, reshapes, reductions, TopK, Resize, ConvTranspose, ScatterND and GridSample -- see tflite_export.SUPPORTED_ONNX_OPS); a node whose op isn't supported raises a clear error naming the op, rather than silently producing a wrong model. Feeding in a simplified model is the point, same as with the other export backends: onnxsim's constant folding turns more of the graph's shape-manipulation subgraphs into plain initializers, which this translator needs at conversion time for things like a Reshape's target shape or a Slice's bounds.

TensorFlow is optional, just like onnxruntime for constant folding and coremltools for Core ML export: it isn't imported unless you actually export to TFLite.

pip install tensorflow

TensorFlow Lite's own op kernels are NHWC-only, while ONNX's conv/pool ops are NCHW; this translator keeps the graph's public tensors in ONNX's NCHW layout and transposes to/from NHWC only around the ops that need it, so no manual layout conversion is required on your part. Graph inputs must have fully static shapes (dynamic axes aren't supported) -- pin them first with --overwrite-input-shape/--test-input-shape if needed.

From the CLI, add --emit-tflite. Passed without a path it writes the model next to the output model with a .tflite extension; pass a path to choose the location:

# writes simplified.onnx and simplified.tflite
onnxsim input.onnx simplified.onnx --emit-tflite
choose the path explicitly, and enable TFLite's default post-training
(dynamic-range) quantization
onnxsim input.onnx simplified.onnx --emit-tflite model.tflite --tflite-optimize

From Python, onnxsim.export_tflite converts a model (typically the output of simplify) and returns the serialized .tflite flatbuffer (bytes), optionally writing it to a file:

import onnx
import onnxsim
model = onnx.load("input.onnx")
model_simp, ok = onnxsim.simplify(model)
assert ok
Return the flatbuffer bytes...
tflite_model = onnxsim.export_tflite(model_simp)
...and/or write it to a file.
onnxsim.export_tflite(model_simp, "model.tflite")

export_tflite accepts an optimizations keyword argument, forwarded to tf.lite.TFLiteConverter.optimizations (e.g. ["DEFAULT"], what --tflite-optimize sets, to enable post-training dynamic-range quantization). See onnxsim/tflite_export.py for the full signature.

A few ops have a correct translation but no TFLite kernel of their own (Atan is one) and fail conversion loudly at the converter. For a model whose only unmappable op is such a CPU-side tail (e.g. BEVFormer box-yaw decoding), pass flex_ops=True (CLI: --tflite-flex) to partition those kernels to TensorFlow Flex on the CPU while everything else stays a TFLite builtin. A Flex model cannot target the Edge TPU (flex_ops is mutually exclusive with --tflite-int8).

A broader-coverage backend: onnx2tf

The built-in translator above covers a practical op subset. For a model that hits an unsupported op, pass backend="onnx2tf" (CLI: --tflite-backend onnx2tf) to route the conversion through onnx2tf instead -- a separate, actively maintained project with far broader op coverage (~200 ops) and years of production hardening across real-world model zoos.

pip install onnx2tf

onnx2tf is a much heavier dependency than the builtin backend needs (it pulls its own TensorFlow, onnxruntime, onnx-graphsurgeon, and a couple dozen small *4onnx helper packages), and it changes the model's public input/output tensor layout to channel-last by default -- it converts every tensor of rank >= 3 to that convention, not just 4-D image tensors, unlike the builtin backend which always keeps ONNX's own declared shapes. Pass onnx2tf's own keep_ncw_or_nchw_or_ncdhw_input_names (a list of input names to keep in their original ONNX layout) as an extra keyword argument if you need specific inputs to keep their original layout.

onnxsim input.onnx simplified.onnx --emit-tflite --tflite-backend onnx2tf

tflite_model = onnxsim.export_tflite(model_simp, backend="onnx2tf")

--tflite-optimize/optimizations only applies to the builtin backend; use onnx2tf's own quantization options (forwarded as extra keyword arguments, e.g. output_integer_quantized_tflite=True) instead. See onnxsim/onnx2tf_export.py for the full signature and onnx2tf's own documentation for its option list.

Running on the Coral Edge TPU

The Edge TPU only runs fully 8-bit quantized models compiled with edgetpu_compiler. onnxsim covers that tail of the pipeline (see onnxsim/edgetpu_export.py):

# 1. full-integer quantization with quantized I/O (TensorFlow required)
onnxsim input.onnx simplified.onnx --emit-tflite model.tflite \
  --tflite-int8 --tflite-io-dtype uint8
2. compile for the Edge TPU (needs the edgetpu_compiler binary)
onnxsim input.onnx simplified.onnx --emit-tflite model.tflite --tflite-edgetpu

--tflite-edgetpu implies --tflite-int8 and writes model_edgetpu.tflite next to the .tflite file (pass a path to choose it), printing per-operator TPU/CPU statuses from the compiler log. Calibration uses uniform-random data (--tflite-calibration-samples N, default 100) unless you pass real representative inputs via the Python API's representative_dataset=. --tflite-edgetpu-check statically checks the simplified model against the Edge TPU requirements (static shapes, supported ops) before converting.

Channel order: --tflite-layout nhwc for larger models

By default the builtin translator keeps public tensors in ONNX's NCHW order and transposes around each conv/pool. That is free for small models (TF folds the interior transposes, leaving just the boundary pair), but the Edge TPU compiler refuses the NCHW entry transpose above modest activation sizes (measured: a 64-channel 32x32 conv fails with large activation tensors, while the identical channel-last graph maps fully; the exit transpose is harmless). --tflite-edgetpu-check warns when a model enters that envelope (4-D activations with 8+ channels and 65536+ elements, inputs and inferred intermediates).

Pass --tflite-layout nhwc (Python: io_layout="nhwc") to carry 4-D tensors channel-last end to end instead: public 4-D I/O changes dimension order to NHWC, but conv/pool/concat emit no transposes at all (verified: the 64ch 32x32 model compiles with every op mapped). Feed NHWC-ordered inputs at inference and when supplying representative_dataset=.

Peak performance on the device itself is characterized in scripts/edgetpu/README.md: a 6-model benchmark suite (pointwise/dense/depthwise/FC workloads) with exact MACs, edgetpu_compiler mapping + on-chip memory stats, and a roofline over USB link speeds — plus a ready-to-run on-device timing script. Short version: the 4 TOPS spec peak is unattainable sustained; expect ~1.6 TOPS for ideal dense compute-bound models on USB3, 0.1–0.4 TOPS for realistic mobile CNNs, and link-bound numbers on USB 2.0.

model_simp, ok = onnxsim.simplify(model)
assert ok
Check first (onnx only, no other dependency)...
report = onnxsim.check_onnx_for_edgetpu(model_simp)
print(report.summary())
...then quantize, compile, and run via LiteRT.
edgetpu = onnxsim.export_edgetpu(model_simp, "model_edgetpu.tflite")
print(edgetpu.compile_result.summary())
out = onnxsim.run_litert("model_edgetpu.tflite", {"x": x_uint8}, use_edgetpu=True)

Inference runs on LiteRT (pip install ai-edge-litert, the successor to tflite-runtime); use_edgetpu=True loads the libedgetpu delegate for on-device execution (see onnxsim.edgetpu_setup_hint() for the runtime/udev setup). Without a device, the same call with use_edgetpu=False runs the quantized model on CPU.

Constant folding on the GPU (CUDA execution provider)

onnxsim constant-folds by running the foldable sub-graphs through ONNX Runtime. By default it uses the CPU execution provider, which is always available and gives deterministic results. For large models it can be much faster to fold on an NVIDIA GPU. Pass providers to simplify to choose the ONNX Runtime execution providers, in priority order:

import onnx
import onnxsim
model = onnx.load(filename)
Fold on the GPU, falling back to CPU for ops CUDA cannot run.
model_simp, check = onnxsim.simplify(
    model, providers=["CUDAExecutionProvider", "CPUExecutionProvider"]
)

On the command line:

# Explicit provider list (priority order):
onnxsim input_onnx_model output_onnx_model \
    --providers CUDAExecutionProvider CPUExecutionProvider
Or the shortcut, equivalent to the line above:
onnxsim input_onnx_model output_onnx_model --cuda

Keeping CPUExecutionProvider last is recommended: ONNX Runtime falls back to it for any operator the GPU provider cannot run. Each provider entry may also be a (name, options) tuple, exactly as onnxruntime.InferenceSession accepts it, for example to pin a specific device_id:

model_simp, check = onnxsim.simplify(
    model,
    providers=[("CUDAExecutionProvider", {"device_id": 1}), "CPUExecutionProvider"],
)

The CUDA execution provider requires the GPU build of ONNX Runtime (pip install onnxruntime-gpu). On AMD ROCm hardware the same providers mechanism reaches ROCMExecutionProvider (pip install onnxruntime-rocm) and MIGraphXExecutionProvider (pip install onnxruntime-migraphx, or the onnxruntime-ep-migraphx plugin on newer ROCm stacks -- see scripts/amd/README.md), for constant folding and -- via step_providers= -- for the QAT/block-finetune/training step graphs as well:

model_simp, check = onnxsim.simplify(
    model, providers=["MIGraphXExecutionProvider", "CPUExecutionProvider"]
)

If you request a provider the installed ONNX Runtime does not offer, onnxsim raises a ValueError listing the available providers instead of silently folding on the CPU. When providers is left unset (the default), folding runs on the CPU.

examples/cuda_feature_tests/ has a notebook exercising this end to end (folding, the CLI, device_id pinning, DLPack CUDA tensors, provider validation) against a real GPU -- open it in Colab and run it by hand whenever you want to check these on an actual NVIDIA GPU; it is not wired into CI.

Constant folding with the AMD NPU (Vitis AI execution provider)

The same providers mechanism works for AMD's Ryzen AI NPU: its ONNX Runtime provider is called VitisAIExecutionProvider, and it partitions the graph into NPU/CPU subgraphs transparently (unsupported ops fall back to the CPU, so keep CPUExecutionProvider last exactly as with CUDA):

model_simp, check = onnxsim.simplify(
    model,
    providers=[
        ("VitisAIExecutionProvider", {"config_file": "vaip_config.json"}),
        "CPUExecutionProvider",
    ],
)

Two differences from CUDA matter. First, the provider never comes from PyPI: the stock onnxruntime / onnxruntime-gpu wheels do not ship it. It comes from AMD's Ryzen AI Software bundle (XRT NPU drivers plus the ryzen_ai venv, which contains the Vitis AI EP build of ONNX Runtime) -- see AMD's Linux install guide and the Vitis AI EP docs. Without that bundle VitisAIExecutionProvider is absent from ort.get_available_providers() and onnxsim raises a ValueError pointing at the Ryzen AI installer (rather than at onnxruntime-gpu). Second, the provider options (config_file for BF16 models, target/xclbin for INT8, cache_dir/cache_key to reuse a compiled model) need the (name, options) tuple form above, which only the Python API offers -- the CLI's --providers takes bare provider names. NPU compilation happens at session creation and can take minutes the first time; the cache options avoid repaying it.

In practice, prefer to keep constant folding itself on the CPU (fold groups are tiny shape/index subgraphs where NPU compile time dwarfs any speedup, and CPU folding is deterministic) and use the NPU for running the full model -- correctness checking (check_n), backend.run_model / backend.Runner, or the QAT/training loops' step_providers=.

Quantized models on the NPU

The EP executes INT8 (and, via config_file, BF16-compiled) graphs; which subgraphs land on the NPU is decided by its own fusion passes. The recommended INT8 recipe is AMD Quark's XINT8 config (pip install amd-quark, no AMD login needed), then onnxsim's Vitis AI legalizer, then the EP with target=X2 (the backend for Strix/KrackanPoint; no xclbin):

from quark.onnx import ModelQuantizer, QConfig
1. Quantize (Quark XINT8: UINT8 activations / INT8 weights, power-of-2 scales).
quantizer = ModelQuantizer(QConfig.get_default_config("XINT8"))
quantizer.quantize_model("fp32.onnx", "int8.onnx", calib_reader)
2. Legalize for the NPU (fixes what the EP can't take -- see below).
import onnxsim
model = onnxsim.legalize_for_vitisai(onnx.load("int8.onnx"))
print(onnxsim.check_vitisai_support(model))  # [] means NPU-safe
3. Run on the NPU (inside the ryzen_ai venv, XRT set up).
import onnxruntime as ort
sess = ort.InferenceSession(
    model.SerializeToString(),
    providers=[("VitisAIExecutionProvider", {"target": "X2"}),
               "CPUExecutionProvider"],
)

Two sharp edges, both measured on Strix Halo / Ryzen AI 1.8:

• Conv without explicit attributes aborts the process. A Conv

• The EP rejects bf16-typed graphs (INVALID_GRAPH); BF16 execution

Profiling the optimization

Simplification alternates a handful of transforms -- shape inference, the onnx-optimizer passes, constant folding and any custom rewriter -- to a joint fixed point. To see where the time and memory go, pass profile to simplify (or --profile on the command line). onnxsim then measures each fixed-point function's wall-clock and CPU duration and the peak resident memory reached while it runs, prints a per-function summary, and writes a Chrome Trace Event Format JSON. Open that file in chrome://tracing or at ui.perfetto.dev to view it as a flame graph: the nested fixed points appear as parent spans and the individual transforms as their children, one box per invocation, annotated with peak RSS and CPU time.

Constant folding's actual work is running the model through ONNX Runtime, so those session runs are profiled too. Each fold group appears under FoldConstant as an OrtSession span, which times running that group's sub-model through the inference executor. This works for every binding, since it wraps the one call site common to both the built-in ONNX Runtime executor and the Python executor that simplify() uses. When the built-in executor runs, the OrtSession span is split further into OrtSessionInit (building the session, where ONNX Runtime loads the graph and usually the dominant cost) and OrtSessionRun (the inference). This makes it easy to see how much of simplification time is spent inside ONNX Runtime versus in shape inference and the optimizer passes.

import onnx
import onnxsim
model = onnx.load(filename)
Write the trace to profile.json (open it in chrome://tracing or ui.perfetto.dev).
model_simp, check = onnxsim.simplify(model, profile="profile.json")

On the command line:

# Give a path, or omit it to use onnxsim_profile.json in the current directory.
onnxsim input_onnx_model output_onnx_model --profile profile.json

The printed summary looks like:

onnxsim profiling summary (per fixed-point function)
-------------------------------------------------------------------------------------
function                calls     wall(ms)      cpu(ms) max wall(ms)    peak(MiB)
-------------------------------------------------------------------------------------
Simplify                    1       260.59       270.93       260.59       112.95
  Pipeline                  3       259.75       269.67       100.76       112.94
    OptAndShape             3       158.63       165.13        53.20       101.43
    FoldConstant            3       100.36       103.78        47.69       112.93
      Optimize              3       112.56       116.99        37.77       101.42
      InferShapes           3        45.46        47.10        15.22        78.68
      OrtSession           12        71.44        74.02        18.31       112.93
        OrtSessionInit     12        58.02        60.11        15.90       112.93
        OrtSessionRun      12         9.85        10.42         2.71       109.10
-------------------------------------------------------------------------------------

(OrtSessionInit/OrtSessionRun show only when the built-in ONNX Runtime executor runs the fold; the Python simplify() path shows just OrtSession.)

calls is how many times a function ran across all fixed-point rounds, cpu(ms) is process CPU time (it can exceed wall time when constant folding runs multiple ONNX Runtime threads), and peak(MiB) is the highest process RSS observed while that function was on the stack (sampled by a lightweight background thread; tune the interval with ONNXSIM_PROFILE_INTERVAL_MS, default 5ms).

Profiling is implemented in onnxsim's C++ core and is driven by the ONNXSIM_PROFILE environment variable (the Python profile argument and the --profile flag just set it), so it also works from the C ABI and the Rust wrapper without any code change:

ONNXSIM_PROFILE=profile.json onnxsim input_onnx_model output_onnx_model

ONNX Runtime's own session profiler

The OrtSession span above times each folding session as a whole. For a finer, per-operator breakdown inside those sessions, turn on ONNX Runtime's own session profiler with ort_profile (or --ort-profile). This flips on SessionOptions.enable_profiling for the ONNX Runtime sessions onnxsim runs while simplifying (the constant-folding sessions, plus the correctness-check runs when check_n > 0), so each one writes ONNX Runtime's detailed per-kernel Chrome trace:

# Write onnxruntime session traces with the given file prefix.
model_simp, check = onnxsim.simplify(model, ort_profile="ort_profile")

onnxsim input_onnx_model output_onnx_model --ort-profile ort_profile

The value is a file prefix: ONNX Runtime writes one _.json per session, so a run that folds in several batches produces several files (open each in chrome://tracing or ui.perfetto.dev). It is independent of profile -- use either, or both together (profile for onnxsim's pipeline, ort_profile for what ONNX Runtime does inside each fold). Like profile, it is driven by an environment variable (ONNXSIM_ORT_PROFILE), so it works from every binding:

ONNXSIM_ORT_PROFILE=ort_profile onnxsim input_onnx_model output_onnx_model

Merging it into onnxsim's trace

Rather than juggling separate files, merge_ort_profile (or --merge-ort-profile) splices ONNX Runtime's per-operator events straight into onnxsim's profile trace, so each OrtSession span gets ONNX Runtime's operator-level detail lined up beneath it on its own onnxruntime track -- one unified flame graph. It implies profile (defaulting to onnxsim_profile.json), and ONNX Runtime's intermediate traces are captured to a temporary directory and removed after merging, so nothing is left behind. This works for every executor, including the Python one simplify() uses:

model_simp, check = onnxsim.simplify(model, profile="profile.json", merge_ort_profile=True)

onnxsim input_onnx_model output_onnx_model --profile profile.json --merge-ort-profile

The merge is also available from the C ABI, Rust and WASM bindings (which fold through the built-in ONNX Runtime executor): set the ONNXSIM_MERGE_ORT_PROFILE environment variable and it is done entirely in onnxsim's C++ core -- no Python needed. It implies ONNXSIM_PROFILE (defaulting to onnxsim_profile.json):

ONNXSIM_MERGE_ORT_PROFILE=1 onnxsim input_onnx_model output_onnx_model

Node-reduction plot

A profile trace also records how many nodes the graph holds right after every round of each fixed-point loop (Optimize, FoldConstant, and Rewrite when a custom_rewriter is given), as NodeCount counter events. onnxsim.profile_plot.plot_node_reduction (or --node-reduction-plot on the command line) turns those into a PNG with one subplot per loop -- node count against round index -- so you can see at a glance how many rounds each loop took and whether it converged (a flat tail) or hit the round cap (ONNXSIM_FIXED_POINT_ITERS, default 50) still descending. It needs matplotlib (pip install onnxsim[plot]):

model_simp, check = onnxsim.simplify(model, profile="profile.json")
from onnxsim.profile_plot import plot_node_reduction
plot_node_reduction("profile.json")  # -> profile.json_node_reduction.png

# Implies --profile if not given explicitly.
onnxsim input_onnx_model output_onnx_model --node-reduction-plot

Custom rewriters

Beyond the built-in optimizer passes, you can plug your own graph rewriting logic into simplification with the custom_rewriter parameter of simplify(). It accepts a callable

Callable[[onnx.ModelProto], Optional[onnx.ModelProto]]

that either returns a rewritten model or mutates the model in place and returns None. The callable runs inside onnxsim's simplification fixed point, interleaved with shape inference, the built-in optimizer and constant folding — so a rewrite can expose new optimization/folding opportunities and vice versa, and the whole pipeline iterates until it converges. onnxsim itself takes no dependency on any particular rewriting library; you bring your own.

Using onnx-rewriter (onnxscript.rewriter)

onnx-rewriter lets you express a subgraph pattern and its replacement as plain Python and have it matched and rewritten anywhere in the model. Install it alongside onnxsim:

pip3 install onnxscript

Then define a rule set and hand it to simplify via custom_rewriter. This example fuses MatMul + Add into a single Gemm:

import onnx
import onnxsim
from onnxscript.rewriter import pattern, rewrite
The subgraph to match: y = MatMul(x, w) + b
def matmul_add_pattern(op, x, w, b):
    return op.Add(op.MatMul(x, w), b)
What to replace it with: y = Gemm(x, w, b)
def gemm_replacement(op, x, w, b):
    return op.Gemm(x, w, b)
rules = pattern.RewriteRuleSet(
    [pattern.RewriteRule(matmul_add_pattern, gemm_replacement)]
)
model = onnx.load("model.onnx")
model_simp, check = onnxsim.simplify(
    model,
    custom_rewriter=lambda m: rewrite(m, pattern_rewrite_rules=rules),
)
assert check, "Simplified ONNX model could not be validated"

Because the rewriter runs every round of the fixed point, the fused Gemm above (and anything it unlocks) is folded and re-optimized together with the rest of the graph.

Skipping the copy when nothing is rewritten

The rewriter runs on every fixed-point round, including the final one where it has nothing left to do — and the fixed point always ends with at least one such no-op round to detect convergence. onnxsim hands the model to your callable as protobuf bytes and parses whatever comes back into a fresh ModelProto, so a rewriter that reports a rewritten model each round pays for that copy even when it changed nothing.

Return False to tell onnxsim that this round rewrote nothing; onnxsim then keeps the model it already has and skips the round-trip. Run the rules through onnx-ir's PassManager — onnxscript.rewriter.RewritePass wraps a rule set as an IR pass — and read the modified flag of the PassResult it returns. That flag is the reliable signal: an IR round-trip can reorder the serialized bytes even when no rule fires, so a byte comparison would falsely report a change.

from onnxscript import ir
from onnxscript.rewriter import RewritePass, pattern
rules = pattern.RewriteRuleSet(
    [pattern.RewriteRule(matmul_add_pattern, gemm_replacement)]
)
rewrite_pass = ir.passes.PassManager([RewritePass(rules)])
def apply_rules(model: onnx.ModelProto):
    model_ir = ir.serde.deserialize_model(model)
    result = rewrite_pass(model_ir)  # ir.passes.PassResult
    if not result.modified:
        return False  # no rule fired this round: skip the copy
    return ir.serde.serialize_model(result.model)
model_simp, check = onnxsim.simplify(model, custom_rewriter=apply_rules)

The plain lambda m: rewrite(m, pattern_rewrite_rules=rules) form still works — it just always returns a model, so onnxsim copies it back every round.

A few things to keep in mind:

• Keep the model schema-valid. After each rewrite onnxsim validates the

• Match the opset your rules target. Convert the model to the opset your

• You are not limited to onnx-rewriter. Any callable works — a hand-written

From the C API and Rust

The custom rewriter lives in onnxsim's C++ core, so the C API and its Rust wrapper expose it too — the model is exchanged as serialized ModelProto bytes across the boundary instead of as an onnx.ModelProto object. In Rust, use simplify_with_rewriter (or simplify_path_with_rewriter) and pass a closure FnMut(&[u8]) -> Result>, E>: return Ok(None) when a round rewrote nothing (onnxsim skips the copy, matching the Python False sentinel), Ok(Some(bytes)) for the rewritten model, or Err(..) to abort.

let simplified = onnxsim::simplify_with_rewriter(
    &model_bytes,
    &onnxsim::Options::new(),
    |bytes: &[u8]| {
        // Decode bytes, rewrite, and return the new bytes — or Ok(None).
        let _ = bytes;
        Ok::<_, onnxsim::Error>(None)
    },
)?;

In C, pass an OnnxsimRewriteFn callback (and an optional matching free callback) to onnxsim_simplify / onnxsim_simplify_path; see onnxsim/capi/onnxsim_c_api.h for the contract. The only binding without it is the standalone CLI, which has no way to carry a user callback.

FunctionProto rules (works in every binding)

custom_rewriter takes a Python callable, so it only works from the Python binding. If inste

... (README truncated for length)