kafSIEM

Edge-ready operations intelligence with an evidence-linked entity graph.

kafSIEM watches Kafka agent traffic and OSINT feeds, materializes entities and relationships in SQLite, and serves analyst workflows through a web desk and typed OpenAPI. Deploy with Docker on a single host. No cluster database required.

OSINT | Operations / Drones | Operations / SCADA

Try in 30 seconds

No Kafka, API keys, or Docker required.

git clone https://github.com/scalytics/kafSIEM.git && cd kafSIEM
make help          # list all targets
make demo-osint    # public OSINT relations demo (fixtures, no Docker)
make demo-ontology # Operations ontology desk
make demo-fusion   # Fusion desk

make demo-osint opens the OSINT console with bundled alert and incident fixtures — cyber, malware, ICS/energy, and terror clusters in the Relations panel. No Kafka or Docker required.

What you get

• Entity graph stored in SQLite: platforms, faults, devices, agents, tasks.

• Analyst desk: flow queue, topology graph, map, replay studio, entity

• Domain packs: YAML ontology for drones and SCADA. Detectors, queries,

• Typed API: /api/v1/... with generated OpenAPI.

• Edge deployment: collector writes, API reads, shared volume. Docker

Who it is for

Unmanned systems teams tracking readiness, sorties, EW events, software lineage, and signoff evidence across a fleet.

SCADA and critical infrastructure teams connecting plants, devices, change audit, alarms, firmware, vulnerabilities, and engineer sessions.

| Pack | Example questions | |------|-------------------| | Drones | What changed since last signoff? Is this fault fleet-wide? Which platforms run software version X? | | SCADA | Which changes lack a work order? Is firmware drifting on safety PLCs? What CVEs affect live devices? |

How it works

KafClaw agents
      |
      v
KafScale / Kafka
      |
      v
kafsiem-collector  --->  /data/agentops.db (+ WAL/SHM)
      |                           |
      | legacy /api/*             v
      +---------------->  kafsiem-api  --->  Caddy + web UI

The collector ingests OSINT sources and, when enabled, consumes Kafka traffic. It is the sole writer for observed operational state. kafsiem-api serves read-only analyst resources and enqueues replay requests. See Architecture.

Operating modes

| Product name | UI_MODE | Purpose | |--------------|-----------|---------| | OSINT | OSINT | Globe-first intelligence, alert feeds, attack-relation clusters | | Operations | AGENTOPS | Kafka agent flow tracking and ontology desk | | Fusion | HYBRID | Operations plus heuristic OSINT correlation |

Set mode via UI_MODE and mounted policy files. User-facing copy uses OSINT, Operations, and Fusion. Runtime env values stay AGENTOPS and HYBRID for compatibility.

Quick start (Docker)

cp .env.example .env
docker compose up --build

UI: http://localhost:8080

Make targets for local HTTP dev:

make dev-start
make dev-stop
make dev-restart
make dev-logs

Remote install

wget -qO- https://raw.githubusercontent.com/scalytics/kafSIEM/main/deploy/install.sh | bash

The installer checks Docker, clones or updates the repo, prompts for profile (OSINT, Operations, or Fusion), site address, and profile-specific settings, then starts GHCR images.

API example

curl -s http://localhost:8080/api/v1/ontology/packs | jq .
curl -s http://localhost:8080/api/v1/flows | jq .
curl -s http://localhost:8080/api/v1/entities/platform/auv-7 | jq .

Full contract: api/openapi.yaml. Client notes: docs/api-clients.md.

Stack context

| Component | Role | |-----------|------| | KafScale | External Kafka transport. kafSIEM consumes from it. | | KafClaw | External agent plane. Emits group envelopes on Kafka topics. | | kafSIEM | Observer, graph store, API, UI, packs (this repository). |

kafSIEM complements enterprise intelligence platforms. It exposes typed entity and graph APIs, pack-defined ontology, and provenance back to source records.

Related projects from the same team: KafGraph (shared knowledge graph for agents) and kshark-core (Kafka connectivity diagnostics, an example of how agents can use our tools to debug Kafka clusters). All projects are listed at scalytics.io/open-source. For a governed, supported platform built on this open core, see Lascaris. Product page: scalytics.io/kafsiem.

Documentation

| Topic | Document | |-------|----------| | Full doc index | docs/README.md | | Architecture | docs/architecture.md | | Operations / Docker / VM | docs/operations.md | | Operations Kafka config | docs/agentops-operator-guide.md | | API clients | docs/api-clients.md | | Drones pack | docs/packs/drones.md | | SCADA pack | docs/packs/scada.md | | OSINT user guide | docs/userguide.md | | Advanced env tuning | docs/advanced-config.md | | Minimal env template | .env.example |

Development

make check
make ci
make docker-build

Node 26.7.0 and npm 11.19.0 are pinned in package.json, .nvmrc, and .node-version. The Go collector powers scheduled feed refresh, Docker runtime, and local collection commands.

Local workflows (make help for the full list):

| Goal | Command | |------|---------| | OSINT relations demo (fixtures) | make demo-osint | | Full local stack (live collector + API) | make dev-start → http://localhost:8080 | | Go collector tests | make go-test | | Full CI gate | make ci or make commit-check |

Browse corroborated clusters: header Relations → filter by attack type (cyber, terror, maritime, conflict). See docs/userguide.md.

License

Apache-2.0. See LICENSE.