cameroncooke/AXe: AXe is a CLI tool for interacting with Simulators using Apple's Private Accessibility APIs.

claude plugin marketplace add cameroncooke/AXe && claude plugin install axe@axe
codex plugin marketplace add cameroncooke/AXe && codex plugin add axe@axe

axe list-simulators                       # find a booted simulator's UDID
export UDID=<UDID>
axe describe-ui --udid "$UDID"            # accessibility tree as JSON
axe tap --label "Continue" --udid "$UDID"
axe type 'Hello world' --udid "$UDID"
axe screenshot --udid "$UDID" --output screen.png

$ axe --output json screenshot --udid "$UDID" --output shot.png
{"command":"screenshot","data":{"result":{"path":"/…/shot.png","pixelHeight":2868,"pixelWidth":1320}},"didError":false,"nextSteps":[],"schema":"axe.output.command-result","schemaVersion":"1"}
$ axe --output json tap --id no-such-element --udid "$UDID"
{"command":"tap","data":{},"didError":true,"error":{"code":"elementNotFound","message":"No accessibility element matched --id 'no-such-element'. …"},"nextSteps":[],"schema":"axe.output.command-result","schemaVersion":"1"}

Feature-specific codes are listed in their sections. JSON Schema (2020-12) files are in schemas/: the command result envelope, each command's data.result (in $defs, keyed by the full command path), and the agent result version 1 (--view full) and version 2. Receipts printed by --verbose are not described. Ignore fields you do not know; AXe can add them within a schema version.

Reading the UI

axe describe-ui --udid "$UDID"                 # the whole tree as JSON
axe describe-ui --point 200,400 --udid "$UDID" # the element at one point

For agents, describe-ui --agent returns compact rows with refs instead; see Agent mode.

Taps, typing and keys

axe tap -x 200 -y 400 --udid "$UDID"
axe tap --id SaveButton --element-type Button --udid "$UDID"
axe tap --label "Done" --wait-timeout 5 --udid "$UDID"   # poll until it appears
axe tap --label "Photo" --hold 1 --udid "$UDID"          # long press an element
axe tap --id Map --count 2 --udid "$UDID"                # double tap
axe tap --id Canvas --touches 2 --udid "$UDID"           # two-finger tap
axe type 'Hello' --udid "$UDID"
printf 'line one\nline two' | axe type --stdin --udid "$UDID"
axe key 40 --udid "$UDID"                                 # Return
axe key-sequence --keycodes 42,42 --udid "$UDID"          # two Backspaces
axe key-combo --modifiers 227 --key 4 --udid "$UDID"      # Cmd+A
axe button home --udid "$UDID"
axe touch -x 100 -y 200 --down --up --delay 1.0 --udid "$UDID"   # long press
axe slider --id volume --value 75 --udid "$UDID"

tap selectors are --id, --label or --value, optionally narrowed with --element-type. On a point or an element, --hold makes a long press, --count 2 a double tap and --touches 2 a two-finger tap (two contacts 40 points apart); these three send real touches and need plain input, without --agent or --input-mode selected. --tap-style chooses automatic (the default: a touch for switches, a simulator tap otherwise), simulator or physical. type uses the US keyboard layout; characters outside it, such as é or £, fail with invalidArguments before anything is typed. Keys take HID usage codes. Buttons are home, lock, side-button, siri and apple-pay.

Swipes, gestures and drawing

axe swipe --start-x 200 --start-y 600 --end-x 200 --end-y 300 --udid "$UDID"
axe gesture scroll-up --udid "$UDID"
axe gesture scroll-down --distance 0.6 --udid "$UDID"               # 60% of the screen height
axe gesture scroll-up --until-label "Privacy" --udid "$UDID"        # scroll until it is on screen
axe gesture scroll-left --inside-id Toolbar --until-label "Share" --udid "$UDID"   # inside one strip
axe gesture swipe-from-top-right-corner --udid "$UDID"              # Control Centre
axe drag --start-x 200 --start-y 300 --end-x 200 --end-y 500 --hold 0.6 --udid "$UDID"  # reorder a row
axe gesture pinch --scale 2 --udid "$UDID"                          # zoom in at the screen centre
axe gesture pinch -x 200 -y 400 --radius 60 --scale 0.5 --udid "$UDID"  # zoom out
axe gesture rotate -x 200 -y 400 --radius 80 --degrees 90 --udid "$UDID"
axe draw 100,400 300,400 --udid "$UDID"                             # a line
axe draw 200,300 300,480 100,480 --close --duration 1.5 --udid "$UDID"  # a triangle

Presets are scroll-up|down|left|right, swipe-from-left|right|top|bottom-edge (--edge-inset moves the start of an edge swipe) and swipe-from-top-right-corner, which opens Control Centre. A scroll preset moves 200 points, or --distance as a fraction of the screen's height or width (0.05 to 0.95). With --until-id or --until-label a scroll preset, or a swipe between your own points, reads the screen, scrolls and reads again until that element is on screen, at most --max-swipes times (default 10, up to 50). An element counts as on screen when a lookup at its centre finds it, so one that sits past the edge of a strip or under another control does not. Each of these scrolls slows to a stop before the finger lifts, so the content moves the length of the scroll and no further; a plain scroll flicks it about as far again. It sends no input when the element is already there, and fails with elementNotFound when the limit is reached or as soon as a scroll moves nothing, which means the content is at its end. --inside-id or --inside-label lays a scroll preset out inside that element instead of the screen, for a list or strip that is only part of the screen (simulators). drag --hold rests the finger on the start before it moves: about 0.6 starts a list reorder or lifts an icon. Presets act on the screen's size in points in the current orientation; --screen-width and --screen-height override it. Two routes do not measure the screen and assume 390×844 unless you pass both: a simulator without the DTUHID touch service, and a gesture step in a batch on a simulator other than the iPhone Duo. Coordinate gestures are pinch, double-tap, rotate, two-finger-tap, two-finger-hold and two-finger-swipe; they need no mode flag. pinch without -x/-y uses the centre of the active screen, and without --radius starts a quarter of the screen's shorter side from it, or closer so that the wider end stays 16 points inside the screen; --scale above 1 zooms in. On a simulator without the DTUHID touch service, coordinate gestures fail with inputUnavailable before input.

draw moves one finger down at the first of 2 to 1,000 x,y points, in straight lines through the rest, and up at the last; --close returns to the first point. The stroke takes --duration seconds (default 1, at most 30). A point outside the active screen fails with invalidArguments before input. drag is the two-point case with explicit move events.

None of these commands proves that the app recognised the gesture; read the screen afterwards.

Batches

axe batch --udid "$UDID" --step "tap --id BackButton" --step "type 'hello'"
axe batch --udid "$UDID" --file steps.txt         # one step per line

Steps are tap, swipe, gesture, draw, touch, type, button, key, key-sequence and key-combo, plus sleep . They run in order in one input session; a failure stops later steps, and earlier steps are not undone. On a simulator, --continue-on-error runs the remaining steps after a failed one; on the iPhone Duo it still stops after a context change or input failure. Success prints ✓ Batch completed successfully (N steps). Agent batches (batch --agent) take JSON instead; see the batch reference.

Screenshots and video

axe screenshot --udid "$UDID" --output screen.png
axe record-video --udid "$UDID" --fps 15 --output recording.mp4      # Ctrl+C to stop
axe stream-video --udid "$UDID" --fps 10 --format mjpeg > stream.mjpeg
axe stream-video --udid "$UDID" --fps 30 --format ffmpeg | \
  ffmpeg -f image2pipe -framerate 30 -i - -c:v libx264 -preset ultrafast output.mp4

stream-video formats are mjpeg (multipart JPEG), raw (4-byte big-endian length, then a JPEG), ffmpeg (concatenated JPEGs) and bgra (packed pixels; a selected-screen stream prints its width and height on stderr, the ordinary simulator stream does not, so pass --screen-id and --screen-uuid when you need them). On a physical device --format h264 streams native H.264 Annex B. Stream bytes go to stdout and diagnostics to stderr; keep them separate. --fps is an upper limit. On any simulator, record-video --follow-active-display follows the active screen through folds and rotations into one fixed-size movie, and --screen-id with --screen-uuid captures one screen and fails if it stops being active; see the recording reference.

Displays, hinge and rotation

$ axe displays --udid "$UDID"
iPhone 18 Pro Max  C096259C-C8FB-42E8-B2E5-8BBE4F8C5AE8
Active  main   440×956 pt @3x  portrait
        id 1   C5182A5C-ECD6-48EE-A5A9-2D7707ED1542

displays waits for a fold or rotation to finish, then shows each screen's role (main on a one-screen device, outer or inner on the iPhone Duo), size in points, scale and orientation, which one is active, and the Duo's hinge angle. JSON is {udid, name, hinge?, activeScreen, otherScreens}. It fails with displayUnavailable while the simulator has not reported a consistent set of screens, as in the first seconds after boot, and with screensDidNotSettle if they keep changing; read again. Pass the active screen's id and uuid as --screen-id and --screen-uuid when a command must refuse to act on any other screen.

On the iPhone Duo:

axe hinge read --udid "$UDID"
axe hinge set tabletop --udid "$UDID"     # 0...180, or closed, tabletop, open
axe rotate left --udid "$UDID"

hinge read returns {udid, hinge: {degrees, mechanicalDegrees, posture}}: degrees is the angle iOS senses, mechanicalDegrees the angle hinge set drives, and posture is closed, partiallyOpen or open. hinge set sends one request, waits up to 8 seconds for the screens to settle (--no-wait skips this) and returns {udid, previousDegrees, hinge, activeScreen?}. rotate left|right works on the Duo and on physical devices and returns {udid, direction, previousRotation?, activeScreen?}; it is not the two-finger gesture rotate. Read the UI again after any of them. Failures: noHinge (one-screen simulator), bootChanged, hingeUnavailable, screensDidNotSettle, rotationFailed, plus the common codes. Duo-specific input, selected-screen input and capture are in the device reference.

Settings

axe settings changes device-wide settings on a booted simulator or a connected iPhone or iPad, through xcrun devicectl (Xcode 27 or later).

$ axe settings show --udid "$UDID"
Appearance            light
Text size             large
Increase Contrast     off
Reduce Motion         off
Reduce Transparency   off
Show Borders          off
Liquid Glass          opacity 0.5
Colour filter         off
VoiceOver             off

axe settings set --udid "$UDID" --appearance dark --text-size accessibility-large
axe settings set --udid "$UDID" --color-filter grayscale --reduce-motion on
axe settings location set --latitude 37.7749 --longitude -122.4194 --udid "$UDID"
axe settings location scenarios --udid "$UDID"
axe settings location scenario "City Run" --udid "$UDID"
axe settings location clear --udid "$UDID"
axe settings status-bar preset screenshot --udid "$UDID"   # also low-battery, no-service, charging
axe settings status-bar set --udid "$UDID" --time 9:41 --wifi-bars 3 --battery-level 100 --battery-state charged
axe settings status-bar clear --udid "$UDID"

set accepts --appearance, --text-size, --increase-contrast, --reduce-motion, --reduce-transparency, --show-borders, --liquid-glass, --liquid-glass-opacity, --color-filter, --color-filter-intensity and --voiceover. It validates every value before changing anything and prints the result as show does. An accessibility text size turns Larger Accessibility Sizes on and a standard size turns it off, so setting the original size restores the device. JSON for show and set is {udid, appearance, textSize, increaseContrast, reduceMotion, reduceTransparency, showBorders, liquidGlass: {style?, opacity}, colorFilter: {enabled, type?, intensity?}, voiceOver}; a setting the device does not report is omitted.

Limits:

• iOS 27.2 offers one Liquid Glass style, so --liquid-glass fails with settingUnsupported; --liquid-glass-opacity works.
• Status bar overrides are simulator-only; on a physical device they fail with settingUnsupported.
• Turning VoiceOver on shows a one-time "VoiceOver Gestures" alert. It stays on screen after VoiceOver is turned off again, and blocks input, until you tap its OK button.
• A simulated location lasts until settings location clear.

Agent mode

Add --agent to describe-ui, tap, type, swipe, drag, key, key-combo, key-sequence, slider, touch, gesture double-tap, gesture pinch and batch for checked, stateful control. A capture returns a snapshot ID and rows ref|actions|role|label|value, such as e9|tap|button|AXe Playground|. An action names a ref from that snapshot; AXe checks that the target is still the control you saw, sends the input, and returns the settled screen as a new capture.

axe describe-ui --agent --view navigation --udid "$UDID"
axe tap --agent --udid "$UDID" --snapshot-id "$SNAPSHOT_ID" --ref e9
axe type 'Ada' --agent --focus --udid "$UDID" --snapshot-id "$SNAPSHOT_ID" --ref e18
axe batch --agent --udid "$UDID" --steps-json '{"snapshotID":"<SNAPSHOT_ID>","steps":[{"action":"tap","ref":"e12"},{"action":"type","ref":"e7","text":"Ada","focus":true}]}'

Results are JSON by default; --output text prints the same result compactly. --view navigation lists controls, reading (the default) adds visible text, and full returns the complete schema version 1. A refusal sends no input and carries a typed code such as screenChanged, targetCovered, keyboardCovered, busy or observationFailed, usually with the current screen. AXe retries inside the call only when the screen or target changed and the target is the same control. Agent mode works on simulators only; a shut-down simulator reports notBooted. AXE_TIMING=1 prints a phase table to stderr after each agent command.

The AXe skill is the guide for agents; the agent reference has every code, limit and output field. --agent needs no model or API key and is separate from axe agent.

Autonomous agent command

axe agent "" --udid turns a natural-language goal into AXe actions with TypeSafe AI's Jev model, one checked action at a time, from the accessibility tree without screenshots. It needs TYPESAFE_API_KEY; values that the goal does not state need an OpenAI-compatible text model (AXE_TEXT_MODEL_API_KEY). Base URLs must use HTTPS, except for localhost, 127.0.0.1 or ::1.

axe agent "Open Settings and turn on Airplane Mode" --udid "$UDID"

See docs/agent.md for configuration, tracing and limitations.

Physical iPhone and iPad

axe list-devices
axe doctor --udid "$DEVICE_UDID"
axe screenshot --udid "$DEVICE_UDID" --output device.png
axe describe-ui --udid "$DEVICE_UDID"
axe tap --udid "$DEVICE_UDID" --id "$ELEMENT_ID" --wait-timeout 15
axe record-video --udid "$DEVICE_UDID" --fps 3 --scale 0.5 --output device.mp4

Pair the device in Xcode, enable Developer Mode and keep it unlocked; AXe never pairs, enables Developer Mode, mounts disk images or unlocks. doctor checks pairing, Developer Mode, connection, developer disk image, lock state and screenshot service, and a failed check ends with Error: is not ready. (it also checks that a simulator is booted). Screenshots, video, accessibility reads, rotate, taps, swipes, drags, draw, gestures, touch --down --up, typing, keys, buttons, batches and settings (except the status bar) work with the hardware UDID.

button lock needs --allow-lock. Agent mode, selected screens, displays, shutdown, hinge, clipboard, slider, side button, Apple Pay and the right-edge swipe preset return unsupportedOnDevice. Physical record-video prints the output path; JSON is {path, pixelWidth, pixelHeight, frames}, and --verbose prints the full receipt. A failed batch step keeps the step's own code. Device-specific codes include deviceLocked, rotationLocked, rotationUnconfirmed, displayOrientationMismatch and deviceStreamReplaced. Read the physical device reference for input rules and video, and docs/device-support.md for the design and limitations.

Simulator shutdown and clipboard

axe shutdown --udid "$UDID"
printf '%s' 'AXe sample' | axe clipboard copy --stdin --udid "$UDID"

shutdown shuts down exactly one simulator by UUID through simctl and prints Shut down (was Booted) or was already shut down; JSON is {udid, initialState, outcome}. clipboard copy stages 1 to 65,536 UTF-8 bytes in one booted simulator's clipboard, verifies them by readback and prints Copied N bytes to the clipboard of (sha256 …); JSON is {udid, bytes, sha256}. It does not paste. Both report actionOutcomeUnknown when the effect could not be confirmed. See the clipboard reference.

Xcode compatibility

AXe supports Xcode 26 and Xcode 27. Xcode 27 simulator automation uses Device Hub; Simulator.app is not required. Release artifacts are built once and run unchanged with either supported Xcode selected. Compatibility was validated with Xcode 26.5 (build 17F42) and iOS 26.5 (23F77), and with Xcode 27 Beta 3 (build 27A5218g) and iOS 27 (24A5380g). axe settings needs Xcode 27.

Development

AXe keeps its own copy of the IDB frameworks it uses, FBControlCore and FBSimulatorControl, in Packages/IDBKit; edit them directly. XcodeGen is required to generate their project.

make build      # rebuilds the IDBKit frameworks only when stale, then swift build
make test       # unit tests
make release-check   # quick release checks in one run; see below
make e2e        # whole Playground suite on dedicated iPhone, iPad and iPhone Duo simulators
make e2e-devices   # opt-in run on connected iPhones and iPads
make bench-quick   # quick A/B timing of direct commands against a baseline payload

make release-check runs everything short of the end-to-end suites and reports each check: released CHANGELOG sections unchanged against origin/main, the script tests, the two skill copies identical, the skill lint, no open items in docs/release-readiness.md, the release packaging dry run, and the unit tests. Every check runs even when one fails. RELEASE_CHECK_SKIP="skill lint,packaging" leaves checks out by name and RELEASE_CHECK_BASE changes the ref.

AXE_BUILD_ARCHS defaults to "arm64 x86_64"; set arm64 for local Duo work. make e2e creates missing simulators, needs Xcode 27.1 or later for the Playground's Duo hinge view, and exits 2 without an iPhone Duo runtime. make e2e-devices exits 1 before building when no device is paired. See Playground testing for simulator overrides and rerunning single groups. To validate a release payload under another Xcode, build the test bundle with that Xcode, then run AXE_BIN_PATH=/path/to/release/axe ./test-runner.sh --tests-only.

Benchmarks

benchmarks/ compares AXe with its best accepted baseline and with FlowDeck and RocketSim on Weather, Contacts and Reminders tasks, recording calls, time and returned tokens. Each run builds and measures the current source.

./benchmarks/run setup     # then set Xcode, a warm simulator and tool paths in benchmarks/config.local.json
./benchmarks/run doctor
./benchmarks/run
./benchmarks/run --tools axe --scenarios weather

Reports stay under benchmarks/history/, which Git ignores; accepted baselines are in benchmarks/baselines.json.

Documentation

Full documentation is available at axe-cli.com/docs.

Disclaimer

AXe is an independent open-source iOS Simulator automation project and is not affiliated with, endorsed by, or associated with Deque Systems or its axe® accessibility products.

Licence

This project is licensed under the MIT License. See LICENSE for details.

Third-party licensing notices, including Meta's IDB MIT attribution, are in THIRD_PARTY_LICENSES.

The keyboard signal detector is adapted from sim-use commit c02295ed under Apache 2.0. Its licence and upstream NOTICE are included in AXe's resource bundle, with the source and changes listed in ADAPTATION.md. AXe's root MIT licence is unchanged.