This Kubernetes Operator is made to easily deploy SeaweedFS onto your Kubernetes cluster.

The operator manages the complete SeaweedFS infrastructure on Kubernetes, including Master servers, Volume servers, and Filer services with S3-compatible API and embedded IAM (Identity and Access Management). This provides a scalable, resilient distributed file system with built-in authentication.

The difference to seaweedfs-csi-driver is that the infrastructure (SeaweedFS) itself runs on Kubernetes as well (Master, Filer, Volume-Servers) and can as such easily scale with it as you need. It is also by far more resilent to failures then a simple systemD service in regards to handling crashing services or accidental deletes.

By using make deploy it will deploy a Resource of type 'Seaweed' onto your current kubectl $KUBECONFIG target (the operator itself) which by default will do nothing unless you configurate it (see examples in config/samples/).

Goals:

• [x] Automatically deploy and manage a SeaweedFS cluster
• [x] Ability to be managed by other Operators
• [x] Compatibility with seaweedfs-csi-driver (deploy it via the SeaweedCSIDriver CR — see CSI_SUPPORT.md)
• [x] Auto rolling upgrade and restart
• [x] Ingress for volume server, filer and S3, to support HDFS, REST filer, S3 API and cross-cluster replication
• [x] IAM (Identity and Access Management) service support for S3 API authentication and authorization
• [ ] Support all major cloud Kubernetes: AWS, Google, Azure
• [x] Scheduled backup and restore: filer metadata snapshots + continuous data mirror to S3, Google Cloud Storage, Azure, Backblaze B2, or a PVC (see BACKUP_SUPPORT.md)
• [ ] Put warm data to cloud storage tier: S3, Google Cloud Storage , Azure
• [x] Grafana dashboard

Installation

Helm

helm repo add seaweedfs-operator https://seaweedfs.github.io/seaweedfs-operator/
helm template seaweedfs-operator seaweedfs-operator/seaweedfs-operator

Note: For versions prior to 0.1.2, the legacy repository URL https://seaweedfs.github.io/seaweedfs-operator/helm can still be used, but new releases will only be published to the main repository URL above.

Upgrading from chart versions <= 0.1.14

Starting in chart version 0.1.15, the seaweeds.seaweed.seaweedfs.com CRD is shipped as a templated resource instead of living in crds/. This lets helm upgrade actually update it — the crds/ directory is install-only in Helm 3.

If you already have the chart installed, run these once before your next helm upgrade so Helm can take over the existing CRD. Look up your release name and namespace first — they must match exactly, or Helm will still refuse to adopt the CRD:

helm list -A | grep seaweedfs-operator
Replace the two values below with the NAME and NAMESPACE you see above.
RELEASE=<release-name>
NAMESPACE=<release-namespace>
kubectl label crd seaweeds.seaweed.seaweedfs.com app.kubernetes.io/managed-by=Helm --overwrite
kubectl annotate crd seaweeds.seaweed.seaweedfs.com \
  meta.helm.sh/release-name=$RELEASE \
  meta.helm.sh/release-namespace=$NAMESPACE --overwrite

The CRD is annotated with helm.sh/resource-policy: keep, so helm uninstall will leave it and your Seaweed resources in place.

If the CRD is managed outside of this chart (e.g., installed cluster-wide via GitOps), set --set crds.create=false on install/upgrade so Helm does not try to own it. Note: helm --skip-crds has no effect here because the CRD lives in templates/, not crds/.

FluxCD

Add the following files to a new directory called seaweedfs-operator under your FluxCD GitRepository (publishing) directory.

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - seaweedfs-operator-namespace.yaml
  - seaweedfs-operator-helmrepository.yaml
  - seaweedfs-operator-helmrelease.yaml

seaweedfs-operator-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: seaweedfs-operator

seaweedfs-operator-helmrepository.yaml

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: seaweedfs-operator
  namespace: seaweedfs-operator
spec:
  interval: 1h
  url: https://seaweedfs.github.io/seaweedfs-operator/

seaweedfs-operator-helmrelease.yaml

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: seaweedfs-operator
  namespace: seaweedfs-operator
spec:
  interval: 1h
  chart:
    spec:
      chart: seaweedfs-operator
      sourceRef:
        kind: HelmRepository
        name: seaweedfs-operator
        namespace: seaweedfs-operator

The webhook is enabled by default. Its server certificate is generated by a pre-install,pre-upgrade Helm hook job, so the seaweedfs-operator-webhook-server-cert secret exists before the operator deployment starts — no manual two-step enable is needed.

Manual

This operator uses kustomize for deployment. Please install kustomize if you do not have it.

By default, the defaulting and validation webhooks are disabled, so make deploy works on any cluster without cert-manager. We strongly recommend enabling the webhooks for production use.

The Kustomize manager deployment uses the same hardened defaults as the Helm chart: it runs as the image's non-root user, applies the RuntimeDefault seccomp profile, disables privilege escalation, drops all Linux capabilities, and uses a read-only root filesystem.

First clone the repository:

git clone https://github.com/seaweedfs/seaweedfs-operator --depth=1

To deploy the operator with webhooks enabled, make sure you have installed the cert-manager(Installation docs: ) in your cluster, then follow the instructions in the config/default/kustomization.yaml file to uncomment all the [WEBHOOK] and [CERTMANAGER] sections (including the one in config/crd/kustomization.yaml). Uncommenting those sections also flips ENABLE_WEBHOOKS to "true" for you via config/default/manager_webhook_patch.yaml, so no separate edit of config/manager/manager.yaml is needed.

Manager image must be locally built and published into a registry accessible from your k8s cluster:

export IMG=<registry/image:tag>
Build and push for amd64
export TARGETARCH=amd64
Optional if you want to change TARGETOS
export TARGETOS=linux
make docker-build
Build and push for arm64
export TARGETARCH=arm64
make docker-build

Afterwards fire up to install CRDs:

make install

Then run the command to deploy the operator into your cluster using Kustomize or Helm:

# if using Kustomize
make deploy
if using Helm
helm install seaweedfs-operator ./deploy/helm

Verify it was correctly deployed:

kubectl get pods --all-namespaces

Which may return:

NAMESPACE                   NAME                                                     READY   STATUS    RESTARTS   AGE
kube-system                 coredns-f9fd979d6-68p4c                                  1/1     Running   0          34m
kube-system                 coredns-f9fd979d6-x992t                                  1/1     Running   0          34m
kube-system                 etcd-kind-control-plane                                  1/1     Running   0          34m
kube-system                 kindnet-rp7wr                                            1/1     Running   0          34m
kube-system                 kube-apiserver-kind-control-plane                        1/1     Running   0          34m
kube-system                 kube-controller-manager-kind-control-plane               1/1     Running   0          34m
kube-system                 kube-proxy-dqfg2                                         1/1     Running   0          34m
kube-system                 kube-scheduler-kind-control-plane                        1/1     Running   0          34m
local-path-storage          local-path-provisioner-78776bfc44-7zvxx                  1/1     Running   0          34m
seaweedfs-operator-system   seaweedfs-operator-controller-manager-54cc768f4c-cwz2k   1/1     Running   0          34m

See the next section for example usage - at this point you only deployed the Operator itself!

You need to also deploy a configuration to get it running (see next section)!

Configuration Examples

Basic SeaweedFS Deployment

For detailed configuration options and examples, see the sample configurations in the config/samples/ directory. For a line-by-line walkthrough of a full cluster, start with config/samples/seaweed_v1_seaweed_annotated.yaml.

Key fields explained

A Seaweed cluster is built from three core components, each run as its own StatefulSet. Every replicas value becomes that many Pods:

• master — coordinates the cluster and assigns volumes. Use 3 replicas for HA, 1 for dev.
• volume — stores the file data on disk. Each replica is a Pod with its own PersistentVolumeClaim(s). Lowering volume.replicas triggers a graceful scale-down: before removing a volume-server Pod the operator evacuates its data to the remaining servers (highest ordinal first, one at a time) and only deletes the Pod once the master confirms the server holds no volumes. A server whose data cannot be moved safely (e.g. no replication-compliant destination) blocks the scale-down rather than risking data loss.
• filer — serves the namespace and the S3/WebDAV/HTTP APIs.

• volume.requests.storage — the size of each volume-server PVC. This is where storage comes from: Kubernetes dynamically provisions a PersistentVolume of this size from the default StorageClass (or volume.storageClassName if set) and binds it to the Pod. To use specific or pre-provisioned disks, see config/samples/seaweed_v1_seaweed_existing_storage.yaml.
• volume.storageAnnotations / volume.storageLabels — stamped onto every generated volume-server PVC (the filer's metadata PVC takes the same under filer.persistence.annotations / .labels). Use these for CSI provisioners that read PVC annotations at provision time — e.g. NetApp Trident's trident.netapp.io/snapshotPolicy and snapshotReserve. Set them at cluster creation: StatefulSet volumeClaimTemplates are immutable, so changing them on a running cluster is not applied automatically — the operator emits a VolumeClaimTemplatesMismatch warning and you must recreate the StatefulSet (e.g. kubectl delete statefulset … --cascade=orphan, which keeps Pods and PVCs) before new PVCs pick up the change. Already-provisioned PVCs keep the metadata they were created with.
• volumeServerDiskCount — number of data disks (PVCs) attached to each volume-server Pod. They are mounted at /data0, /data1, … and passed to the volume server as -dir. Total PVCs = volume.replicas × volumeServerDiskCount. Leave at 1 unless a node exposes multiple disks.
• master.volumeSizeLimitMB — the max size of a single logical volume file before the master allocates a new one (1024 = 1 GiB per file). This is not the cluster capacity and not the PVC size — total capacity is driven by the volume servers' disks.
• master.persistence — a volume for the master's -mdir, off by default. That directory holds the raft log and snapshots, and with them the cluster's identity (its TopologyId); without it the master runs on the container's writable layer and mints a new identity every time all masters restart together. Volume IDs survive regardless — the master rebuilds MaxVolumeId from volume-server heartbeats — so this is about identity, not data. Takes the same fields as filer.persistence. existingClaim is one volume for the whole StatefulSet, so it is only accepted for a single master — every master keeps its raft state under the same subdirectory of -mdir, and replicas sharing one volume would overwrite each other. Turn it on at cluster creation: it adds a volumeClaimTemplate, and those are immutable, so an existing StatefulSet has to be recreated (kubectl delete statefulset … --cascade=orphan) before it takes.

• master.ipBind / volume.ipBind / filer.ipBind — the address those components bind their listeners to (weed -ip.bind). Defaults to 0.0.0.0, matching the official SeaweedFS Helm chart. The operator advertises each Pod's headless-service FQDN via -ip, and weed binds to whatever -ip names unless told otherwise — which means resolving that record milliseconds into container start. On a cold start CoreDNS has not propagated it yet, so the process exits and every master/volume/filer Pod restarts once. Binding to the wildcard needs no DNS and does not change what is advertised to the master, so cluster registration and peer discovery are unaffected. Set an address to bind a single interface, or "" to restore weed's own behavior of binding to -ip.
• hostSuffix — optional. Creates a single all-in-one Ingress exposing the cluster under filer., s3., and -volume-. (requires an Ingress controller). Omit it for in-cluster-only access, or use the per-component ingress: blocks for finer control.
• master.config / filer.config — raw TOML dropped verbatim into that component's config file (master.toml / filer.toml). Yes, you can paste an existing SeaweedFS filer config here — for example to point the filer's metadata store at Postgres/MySQL/Redis instead of local leveldb2.
• master.configSecret / filer.configSecret — the same TOML, but read from an existing Secret instead of the CR, so credentials in it (a metadata-store password, remote storage keys) stay out of kubectl get seaweed -o yaml, etcd and Git. The referenced key is mounted as master.toml / filer.toml, which lets External Secrets Operator, Sealed Secrets or SOPS own and rotate it. Set one of config or configSecret per component, not both — the API rejects it.

filer:
    configSecret:
      name: seaweedfs-filer-config   # Secret in the same namespace
      key: filer.toml                # key inside it; any name works

Rotating the Secret updates the mounted file in place, but SeaweedFS reads its TOML only at startup — restart the component's Pods (kubectl rollout restart statefulset/-filer) for a change to take effect. Switching a component from config to configSecret also deletes the ConfigMap the operator generated for the inline config, so the plaintext copy does not linger in the namespace.

To run with a cloud bucket as remote storage (Cloud Drive) backed by a local cache, see config/samples/seaweed_v1_seaweed_remote_storage.yaml.

Bare-metal volume servers (DaemonSet + node-local disks)

By default volume servers run as a StatefulSet with one or more dynamically-provisioned PVCs per Pod. On on-prem / bare-metal clusters you often instead want **one volume server per node, writing straight to that node's physical disks**. Two volume fields enable this:

• volume.kind: DaemonSet — runs exactly one volume server on every node

• volume.hostPath — a list of node-local directories to use as data

hostPath is required for kind: DaemonSet (DaemonSets cannot use volumeClaimTemplates) and the operator rejects the combination otherwise. It also works with a StatefulSet — pair it with node anti-affinity so two replicas never share a host directory.

spec:
  volume:
    kind: DaemonSet
    replicas: 0            # ignored for DaemonSet
    hostPath:
      - path: /mnt/disks/ssd0
        maxVolumeCount: 100
      - path: /mnt/disks/ssd1
    nodeSelector:
      seaweedfs.com/storage: "true"
    tolerations:
      - key: seaweedfs.com/storage
        operator: Exists
        effect: NoSchedule

See config/samples/seaweed_v1_seaweed_hostpath_daemonset.yaml for a full example. For rack/datacenter-aware placement across multiple volume groups, see TOPOLOGY_SUPPORT.md.

IAM Support

The operator supports IAM (Identity and Access Management) for S3 API authentication. IAM is embedded in the S3 server and runs on the same port (8333) as the S3 API.

For complete IAM configuration details, OIDC setup, and troubleshooting, see IAM_SUPPORT.md.

Example Configuration

apiVersion: seaweed.seaweedfs.com/v1
kind: Seaweed
metadata:
  name: seaweed-sample
  namespace: default
spec:
  image: chrislusf/seaweedfs:latest
  volumeServerDiskCount: 1
  hostSuffix: seaweed.abcdefg.com
  master:
    replicas: 3
    volumeSizeLimitMB: 1024
  volume:
    replicas: 1
    requests:
      storage: 2Gi
  filer:
    replicas: 2
    config: |
      [leveldb2]
      enabled = true
      dir = "/data/filerldb2"
  # Standalone S3 gateway — the preferred way to expose the S3 API. Creates a
  # "seaweed-sample-s3" Service on port 8333 (IAM is embedded on the same port
  # by default). See "S3 API" below.
  s3:
    replicas: 1

For more examples, see the config/samples/ directory:

• seaweed_v1_seaweed.yaml - Basic deployment with the standalone S3 gateway
• seaweed_v1_seaweed_annotated.yaml - Basic deployment with every field explained
• seaweed_v1_seaweed_existing_storage.yaml - Specific StorageClass / pre-provisioned PVs for local block storage
• seaweed_v1_seaweed_hostpath_daemonset.yaml - Bare-metal volume servers as a DaemonSet on node-local hostPath disks
• seaweed_v1_seaweed_remote_storage.yaml - Local cache plus a remote cloud bucket (Cloud Drive)
• seaweed_v1_seaweed_with_iam_embedded.yaml - S3 with embedded IAM
• seaweed_v1_seaweed_with_tls.yaml - mTLS between components via cert-manager
• seaweed_v1_seaweed_ingress_tls.yaml - Expose the S3 API and filer over TLS via per-component Ingress

S3 API

There are two ways to expose the S3 API. Prefer the standalone S3 gateway for new clusters.

Standalone S3 gateway (recommended) — set the top-level spec.s3 block. The operator runs S3 as its own stateless Deployment and puts a dedicated Service in front of it named -s3 (for a cluster named seaweed1, the Service is seaweed1-s3), listening on port 8333:

spec:
  filer:
    replicas: 2          # the gateway dials the filer, so it must be enabled
  s3:
    replicas: 1          # stateless — scale freely
    # port: 8333         # override the default S3 port
    # httpsPort: 8334    # add an HTTPS listener (requires tlsSecret, see below)
    # httpsOnly: true    # drop the plain HTTP listener
    # tlsSecret:
    #   name: seaweed-s3-server-tls
    # domainName: s3.example.com   # for virtual-hosted-style buckets
    # metricsPort: 9327  # enable the Prometheus listener + a ServiceMonitor
    configSecret:        # optional: S3 identities (the -s3.config equivalent)
      name: my-s3-config
      key: seaweedfs_s3_config.json
    # service:           # optional: change the Service that fronts the gateway
    #   type: LoadBalancer
    #   annotations: {}
    # ingress:           # optional: per-component Ingress
    #   enabled: true

Reach it in-cluster at http://-s3..svc:8333. To expose it externally, set spec.s3.service.type: LoadBalancer, add an spec.s3.ingress block (see Exposing the cluster via Ingress (TLS) below), or use the top-level hostSuffix, which publishes s3. over HTTP.

The operator also publishes the in-cluster endpoints it creates into status: status.master.endpoint, status.filer.endpoint, status.admin.endpoint, status.s3.endpoint and status.sftp.endpoint, each alongside a service field naming the Service (e.g. status.s3.endpoint is http://-s3..svc:8333, or https:// when the gateway serves HTTPS). Volume servers and workers front no single client Service, so their status carries only replica counts.

Native HTTPS on the standalone gateway — set httpsPort and tlsSecret together to have the gateway itself serve HTTPS, without an Ingress:

spec:
  s3:
    replicas: 1
    httpsPort: 8334
    tlsSecret:
      name: seaweed-s3-server-tls   # must contain tls.crt and tls.key

The operator mounts the Secret's tls.crt and tls.key read-only at /etc/sw-s3-tls, passes -port.https, -cert.file and -key.file to weed s3, and adds an s3-https port to the -s3 Service. Clients then use https://-s3..svc:8334.

• Plain HTTP keeps serving on port (8333) alongside HTTPS, and stays

• The Secret is any kubernetes.io/tls-shaped Secret: one you create with

• The gateway re-reads the certificate from disk every 5 hours by default, so

spec:
    s3:
      env:
        - name: WEED_TLS_CERT_REFRESH_INTERVAL
          value: 10m

• Clients must trust whichever CA issued tls.crt. With a public CA (e.g.

• This is independent of spec.tls. The spec.tls mTLS material still secures

• The CRD rejects httpsPort without tlsSecret (and vice versa), and an

HTTPS-only — add httpsOnly: true to serve S3 over HTTPS alone:

spec:
  s3:
    replicas: 1
    httpsPort: 8334
    httpsOnly: true
    tlsSecret:
      name: seaweed-s3-server-tls

The gateway then listens only on httpsPort, the Service exposes only s3-https, and the probes call /status over HTTPS (kubelet does not verify the certificate). The gRPC port stays at port + 10000. httpsOnly requires httpsPort and tlsSecret, and the CRD rejects it together with spec.s3.ingress or spec.hostSuffix, both of which route to the HTTP port.

Embedded filer S3 (deprecated) — the older spec.filer.s3.enabled: true runs S3 inside every filer pod and exposes it as the filer-s3 port on the -filer Service. There is no -s3 Service in this mode. It is retained for backward compatibility but deprecated; the admission webhook rejects setting both paths at once and warns when the embedded path is used. Migrate by moving the config to the top-level spec.s3 block above.

IAM (S3 authentication) is embedded in the S3 server and runs on the same port. See IAM_SUPPORT.md.

Exposing the cluster via Ingress (TLS)

By default the operator only creates in-cluster Services. There are two ways to expose components (S3 API, filer, master/admin UIs, volume servers) outside the cluster through an Ingress controller. Both require an Ingress controller (ingress-nginx, Traefik, …) and DNS pointing at it.

• hostSuffix — the legacy all-in-one helper. One Ingress under

• Per-component ingress: blocks — the recommended path, and the only

The ingress: block is available on master, volume, filer, filer.s3Ingress (the filer's embedded S3 port), admin, and the standalone s3 and sftp gateways. Every block shares the same fields:

| Field | Description | | --- | --- | | enabled | Create the Ingress for this component. | | host | Hostname the Ingress matches (required when enabled). | | className | IngressClassName, e.g. nginx. | | path | Path prefix to serve. Defaults to /. | | annotations | Controller-specific annotations (cert-manager issuer, nginx body size, …). | | labels | Extra labels on the Ingress, e.g. type: sharded for an OpenShift IngressController routeSelector. The operator's own app.kubernetes.io/name, /instance, /component, and /managed-by values take precedence. | | tls | List of {hosts, secretName} — terminates TLS using a kubernetes.io/tls Secret. |

To reach the S3 API over an https:// URL, enable the S3 API and give it a TLS Ingress. With the filer-embedded S3 (filer.s3.enabled):

spec:
  filer:
    replicas: 1
    s3:
      enabled: true            # S3 API on port 8333 (IAM embedded)
    s3Ingress:
      enabled: true
      className: nginx
      host: s3.seaweed.example.com
      annotations:
        # cert-manager issues the cert into secretName below; omit if you
        # created the TLS Secret by hand.
        cert-manager.io/cluster-issuer: letsencrypt-prod
        nginx.ingress.kubernetes.io/proxy-body-size: "0"   # allow large S3 PUTs
      tls:
        - hosts: [s3.seaweed.example.com]
          secretName: seaweed-s3-tls

If you are not using cert-manager, create the TLS Secret yourself and drop the issuer annotation:

kubectl create secret tls seaweed-s3-tls --cert=tls.crt --key=tls.key

Then point any S3 client at the TLS endpoint:

aws --endpoint-url https://s3.seaweed.example.com s3 ls

Prefer the standalone S3 gateway (scales independently of the filer) by putting the same ingress: block under the top-level s3: instead of filer.s3Ingress — the fields are identical. See config/samples/seaweed_v1_seaweed_ingress_tls.yaml for a complete manifest.

Note: this Ingress TLS terminates HTTPS at the Ingress controller. It is

separate from spec.tls, which provisions cert-manager-issued mTLS

between SeaweedFS components (master/volume/filer gRPC). To have the

standalone gateway serve HTTPS itself, see spec.s3.httpsPort under

S3 API above.

TLS Between Components (cert-manager)

The operator can provision mTLS between the SeaweedFS components (master, volume, filer, S3) using cert-manager, which must be installed in the cluster. When spec.tls.enabled is true, the operator creates a cert-manager Certificate covering every component's headless Service and renders a security.toml that wires mTLS into every gRPC endpoint. If the cert-manager CRDs are absent, the operator records a condition on the Seaweed CR and leaves TLS off instead of failing.

By default (no issuerRef) the operator provisions a self-signed Issuer + CA Certificate + CA Issuer chain owned by the Seaweed CR — no external issuer required:

spec:
  tls:
    enabled: true

To sign the server certificate from a cert-manager Issuer or ClusterIssuer you already manage, set issuerRef and the operator skips the self-signed chain:

spec:
  tls:
    enabled: true
    issuerRef:
      name: my-ca-issuer
      kind: ClusterIssuer      # or Issuer (the default)
      group: cert-manager.io   # default

Note: this spec.tls block configures mTLS between SeaweedFS components. It is

independent of the operator's own admission-webhook serving certificate, which is

covered under Installation → Manual.

See config/samples/seaweed_v1_seaweed_with_tls.yaml for a full example.

JWT Signing (security.toml)

spec.securityConfig.jwtSigning controls the [jwt.*] sections of the security.toml the operator renders, mirroring the official chart's global.seaweedfs.securityConfig.jwtSigning. Each flag turns one section on; the operator generates an HMAC key for it, keeps that key stable across reconciles, and stores the file in a Secret (-security-config) mounted at /etc/sw-security on every component.

spec:
  securityConfig:
    jwtSigning:
      volumeWrite: true     # [jwt.signing]
      volumeRead: false     # [jwt.signing.read]
      filerWrite: false     # [jwt.filer_signing]
      filerRead: false      # [jwt.filer_signing.read]
      expiresAfterSeconds:  # 0 keeps weed's defaults: 10s writes, 60s reads
        volumeWrite: 0
        volumeRead: 0
        filerWrite: 0
        filerRead: 0

| flag | section | what it enforces | |---|---|---| | volumeWrite | [jwt.signing] | the master signs an upload token per assigned file id; the volume server rejects unsigned writes | | volumeRead | [jwt.signing.read] | the volume server rejects unsigned reads; clients that read chunks directly (weed mount, the CSI driver) need the same key | | filerWrite | [jwt.filer_signing] | the filer rejects unsigned HTTP writes, and its IAM gRPC service requires an admin Bearer token | | filerRead | [jwt.filer_signing.read] | the filer rejects unsigned HTTP reads; the S3 gateway signs its own, plain HTTP GETs do not |

Every flag defaults to false, and a section that is not rendered is not enforced — seaweedfs treats a missing key as "do not check". If no flag is set and spec.tls is off, no security.toml is created at all.

Notes worth knowing before turning these on:

• The filer UI's upload form does not sign its requests, so it stops

• With filerRead on, plain HTTP GETs against the filer (including the filer

• With filerWrite on, the operator signs its own IAM gRPC calls (S3Identity,

• Toggling a flag changes a seaweed.seaweedfs.com/jwt-signing pod annotation,

• Turning a section off and back on mints a new key: the operator only

Upgrading from an operator version before this field existed: the old

behavior was to render [jwt.filer_signing] unconditionally whenever a filer

or admin was in spec, with no way to switch it off. That is now opt-in — set

filerWrite: true to keep it.

See config/samples/seaweed_v1_seaweed_jwt_signing.yaml for a full example.

Prometheus metrics

Set metricsPort on a component to serve its metrics at /metrics on that port. When the Prometheus Operator CRDs are installed, the operator also creates a ServiceMonitor named - for it. To scrape the metrics another way, such as with a PodMonitor of your own, keep the ports and turn the ServiceMonitors off:

spec:
  serviceMonitor:
    enabled: false       # defaults to true
  master:
    metricsPort: 9327
  filer:
    metricsPort: 9327

The operator then deletes the ServiceMonitors it created earlier. It does the same for a component whose metricsPort is removed. A ServiceMonitor the Seaweed does not control is never touched.

The operator checks for the ServiceMonitor CRD once and caches the answer, so restart it after installing the Prometheus Operator.

Declarative Buckets

The Bucket CRD (seaweed.seaweedfs.com/v1) provisions S3 buckets inside an existing Seaweed cluster. It mirrors the surface of weed shell s3.bucket.* and fs.configure so the same operations users run manually become declarative manifests that GitOps tools (FluxCD, ArgoCD, OpenTofu) can apply and reconcile.

A minimal bucket:

apiVersion: seaweed.seaweedfs.com/v1
kind: Bucket
metadata:
  name: photos
  namespace: media
spec:
  clusterRef:
    name: seaweed1
    namespace: default

Supported per-bucket configuration:

• versioning: Off (default), Enabled, Suspended. Once enabled,

• objectLock: enable S3 Object Lock. Requires versioning: Enabled

• quota: cap total stored size with resource.Quantity (e.g. 100Gi)

• owner / access: bind an existing IAM identity as bucket owner and

• placement: pin replication, disk type, default TTL, fsync, WORM,

• reclaimPolicy: Retain (default) leaves data untouched on CR

• adoptExisting: false (default) refuses a pre-existing bucket of

• Cross-namespace clusterRef is denied by default: it resolves only

CEL admission validations enforce: S3-compliant bucket-name regex, the objectLock ↔ versioning interlock, immutability of objectLock once enabled, and the "no return to Off" versioning transition rule.

See the config/samples/seaweed_v1_bucket*.yaml files for end-to-end examples (minimal, full-featured, object lock, cross-namespace).

Usage stats

The operator periodically refreshes status.usage (object count, total bytes, last-updated timestamp) on every Bucket by issuing one collection.list call per Seaweed cluster and patching each bucket's status. The cadence is configurable via the --bucket-usage-refresh-interval flag (default 5m). Set to 0 to disable. The loop is leader-elected so HA deployments do not duplicate work.

Usage stats are best-effort observation — they do not block reconcile or affect quota enforcement (the underlying S3 quota check on writes is authoritative). When a Bucket has not been successfully reconciled yet (status.bucketName empty), it is skipped until the main reconcile loop has provisioned it.

COSI coexistence

The SeaweedFS COSI driver also creates buckets via the upstream objectstorage.k8s.io/v1alpha1 API. The two are complementary: COSI is the right choice when an application needs a bucket-claim lifecycle bound to a workload, while the Bucket CRD is the right choice for cluster- or platform-team-owned buckets with quotas, placement, and IAM grants. The controller never adopts or modifies a bucket created by the COSI driver unless adoptExisting: true explicitly opts in — collisions are surfaced as BucketAlreadyExists in status rather than silently overwriting.

Bucket lifecycle policies

The BucketLifecyclePolicy CRD (seaweed.seaweedfs.com/v1) manages the S3 lifecycle configuration of a bucket declaratively, so object expiration and cleanup rules live in Git instead of being applied through the S3 API by hand.

apiVersion: seaweed.seaweedfs.com/v1
kind: BucketLifecyclePolicy
metadata:
  name: expire-logs
  namespace: media
spec:
  bucketRef:
    name: photos
  rules:
    - id: expire-archived
      prefix: archived/
      status: Enabled
      expiration:
        days: 90
    - id: cleanup-incomplete-uploads
      status: Enabled
      abortIncompleteMultipartUpload:
        daysAfterInitiation: 7

• bucketRef points at a Bucket in the same namespace; the cluster and

• Each rule needs an id and at least one action: expiration

• A single policy owns a bucket's whole lifecycle configuration — the

• reclaimPolicy: Delete (default) removes the rules from the bucket

• Taking over a bucket that used the older per-path day-TTL lifecycle

See config/samples/seaweed_v1_bucketlifecyclepolicy.yaml for a full example.

Declarative IAM (identities, credentials, policies)

Four CRDs (seaweed.seaweedfs.com/v1) manage the S3 IAM objects of a Seaweed cluster declaratively, so users, access keys, and permissions can be GitOps-managed alongside the Bucket CRD. They drive the cluster's embedded IAM service (the IAM gRPC API on the filer — see IAM_SUPPORT.md) and mirror weed shell's s3.user., s3.accesskey., and s3.policy* commands.

Unlike Bucket (which holds data and defaults to reclaimPolicy: Retain), these resources are pure configuration: the CR is the source of truth, so they default to reclaimPolicy: Delete — deleting the CR removes the underlying IAM object. Set reclaimPolicy: Retain to opt out.

• S3Identity — an IAM user. Created with no credentials by default;

apiVersion: seaweed.seaweedfs.com/v1
  kind: S3Identity
  metadata: { name: alice, namespace: default }
  spec:
    seaweedRef: { name: seaweed1 }

• S3Credentials — an access key / secret key pair for an identity,

apiVersion: seaweed.seaweedfs.com/v1
  kind: S3Credentials
  metadata: { name: alice-creds, namespace: default }
  spec:
    seaweedRef: { name: seaweed1 }
    identityRef: { name: alice }
    secretRef: { name: alice-s3-secret }   # accessKeyField/secretKeyField default to accessKey/secretKey

• S3Policy — an IAM policy. Author it as structured statements

apiVersion: seaweed.seaweedfs.com/v1
  kind: S3Policy
  metadata: { name: rw-uploads, namespace: default }
  spec:
    seaweedRef: { name: seaweed1 }
    statements:
      - effect: Allow
        actions: [s3:GetObject, s3:PutObject, s3:DeleteObject]
        resources: [my-bucket/uploads/*]

• S3PolicyBinding — attaches a policy to a set of identities. The

apiVersion: seaweed.seaweedfs.com/v1
  kind: S3PolicyBinding
  metadata: { name: alice-uploads, namespace: default }
  spec:
    seaweedRef: { name: seaweed1 }
    policyRef: { name: rw-uploads }
    subjects:
      - { kind: S3Identity, name: alice }
      - { kind: S3Identity, name: bob }

IAM user and policy names are global to the cluster while these CRs are namespaced:

• Name conflicts — when CRs in different namespaces claim the same

• Reference resolution — identityRef, policyRef, and subjects

S3Credentials and S3PolicyBinding wait (status Pending) until the identity / policy they reference exists, so apply order does not matter. As with Bucket, a cross-namespace seaweedRef (and the S3Credentials secretRef) is denied by default and requires a ResourceReferenceGrant in the target namespace. When the filer enforces jwt.filer_signing.key (rendered into the cluster's security.toml by securityConfig.jwtSigning.filerWrite), the operator reads that key and signs its own IAM gRPC calls with it, so authenticated filers are handled automatically.

See the config/samples/seaweed_v1_s3*.yaml files for end-to-end examples.

Cross-namespace references (ResourceReferenceGrant)

By default a SeaweedFS resource may only reference resources in **its own namespace*. A reference that crosses namespaces — a Bucket/S3 seaweedRef/clusterRef pointing at a Seaweed in another namespace, or an S3Credentials secretRef pointing at a Secret in another namespace — is refused until the target namespace publishes a ResourceReferenceGrant that allows it. This mirrors the Gateway API ReferenceGrant: the namespace that owns the resource being pointed at — not the requester — decides who may reach in.

The grant lives in the namespace of the resource being referenced. Its spec.from lists the trusted sources — each names a {group, kind} plus the source namespaces, given either as an exact namespace or as a namespaceSelector (exactly one per entry) — and its spec.to lists the {group, kind, name?} referents in that namespace (omit name to allow every resource of that kind). A reference is allowed when it matches at least one from and one to entry.

# In the cluster's namespace: let the "media" namespace's Buckets and
S3Credentials reference the Seaweed cluster "prod".
apiVersion: seaweed.seaweedfs.com/v1
kind: ResourceReferenceGrant
metadata:
  name: allow-media
  namespace: seaweedfs
spec:
  from:
    - { group: seaweed.seaweedfs.com, kind: Bucket, namespace: media }
    - { group: seaweed.seaweedfs.com, kind: S3Credentials, namespace: media }
  to:
    - { group: seaweed.seaweedfs.com, kind: Seaweed }   # any Seaweed here

For environments where source namespaces are created on demand (per tenant, per PR, ...) and cannot be enumerated ahead of time, a from entry may select namespaces by label with namespaceSelector instead of naming one. Every namespace whose labels match is trusted, so labelling a freshly created namespace grants access without editing the grant. An empty selector ({}) matches all namespaces.

# Trust every namespace labelled seaweedfs-access=true to reference any
Seaweed cluster in this namespace.
apiVersion: seaweed.seaweedfs.com/v1
kind: ResourceReferenceGrant
metadata:
  name: allow-labeled-buckets
  namespace: seaweedfs
spec:
  from:
    - group: seaweed.seaweedfs.com
      kind: Bucket
      namespaceSelector:
        matchLabels: { seaweedfs-access: "true" }
  to:
    - { group: seaweed.seaweedfs.com, kind: Seaweed }

While a required grant is missing the referencing resource stays Pending (Bucket surfaces condition ClusterRefForbidden=True; the S3* kinds surface ReferenceGranted=False) and reconciles to ready automatically once the grant is created — at which point the condition is cleared.

Enforcement is reconcile-time and eventually consistent (like every cross-resource dependency here, and like Gateway API): revoking a grant stops the operator from (re)provisioning the reference on the next reconcile, but does not retroactively tear down objects already provisioned under it. Deleting a resource is never blocked by a missing grant, so revoking one cannot strand a finalizer. See config/samples/seaweed_v1_resourcereferencegrant.yaml.

CSI Driver (mount volumes as PersistentVolumes)

The operator can also deploy the seaweedfs-csi-driver so that pods mount a SeaweedFS filer as ordinary PersistentVolumes (a POSIX FUSE mount), including ReadWriteMany volumes shared across nodes. This is the filesystem alternative to the S3 API above.

A CSI driver is node-global, so it is managed through its own opt-in SeaweedCSIDriver resource rather than a field on the Seaweed CR, and the controller is off by default — enable it with ENABLE_CSI_DRIVER=true on the operator manager, or csiDriver.enabled: true with the Helm chart. The driver can mount an operator-managed cluster (seaweedRef, grant-gated across namespaces) or any external filer (filerAddress):

apiVersion: seaweed.seaweedfs.com/v1
kind: SeaweedCSIDriver
metadata:
  name: seaweedfs
spec:
  seaweedRef:
    name: seaweed1
  storageClass:
    name: seaweedfs
    parameters:
      replication: "000"

Pods then request a PVC against the seaweedfs StorageClass. See CSI_SUPPORT.md for the full guide, API reference, and the list of managed objects. Example: config/samples/seaweed_v1_seaweedcsidriver.yaml.

Scheduled admin scripts (AdminScript)

The AdminScript CRD (seaweed.seaweedfs.com/v1) runs a weed shell script on a cron schedule against a cluster — for recurring maintenance such as volume.balance, volume.fix.replication, ec.encode, or volume.vacuum. The operator reconciles each AdminScript into a native Kubernetes CronJob in the same namespace, owned by the CR (so deleting the AdminScript removes the CronJob). Each run pipes the script into weed shell -master= (and -filer= when the cluster runs a filer). The run pod mirrors the cluster's admin/worker pods — it uses the cluster image and, when the cluster has mTLS enabled, mounts the same security.toml/TLS material so the s

... (README truncated for length)