Profile
Back to NewsBack
GitHub Trending 9 min
Reader Mode
sivchari/kumo: A lightweight AWS service emulator written in Go

sivchari/kumo: A lightweight AWS service emulator written in Go

7 hours ago

kumo logo

Go Version License Release Lint Integration Tests

A lightweight AWS service emulator written in Go.
Works as both a CI/CD testing tool and a local development server with optional data persistence.

Features

  • No authentication required - Perfect for CI environments
  • Single binary - Easy to distribute and deploy
  • Docker support - Run as a container
  • Lightweight - Fast startup, minimal resource usage
  • AWS SDK v2 compatible - Works seamlessly with Go AWS SDK v2
  • Optional data persistence - Survive restarts with KUMO_DATA_DIR

Supported Services (82 services)

Storage

| Service | Description | |---------|-------------| | DynamoDB | NoSQL database | | DynamoDB Streams | DynamoDB change data capture | | EBS | Block storage | | ElastiCache | In-memory caching | | Glacier | Archive storage | | MemoryDB | Redis-compatible database | | S3 | Object storage | | S3 Control | S3 account-level operations | | S3 Tables | S3 table buckets |

Compute

| Service | Description | |---------|-------------| | Batch | Batch computing | | EC2 | Virtual machines | | Elastic Beanstalk | Application deployment | | Lambda | Serverless functions |

Container

| Service | Description | |---------|-------------| | ECR | Container registry | | ECS | Container orchestration | | EKS | Kubernetes service |

Database

| Service | Description | |---------|-------------| | DocumentDB | MongoDB-compatible database | | Neptune | Graph database | | RDS | Relational database service | | Redshift | Data warehousing |

Messaging & Integration

| Service | Description | |---------|-------------| | EventBridge | Event bus | | Firehose | Data delivery | | Kinesis | Real-time streaming | | MQ | Message broker (ActiveMQ/RabbitMQ) | | MSK (Kafka) | Managed streaming for Kafka | | Pipes | Event-driven integration | | SNS | Pub/Sub messaging | | SQS | Message queuing |

Security & Identity

| Service | Description | |---------|-------------| | ACM | Certificate management | | Cognito | User authentication | | IAM | Identity and access management | | KMS | Key management | | Macie | Data security and privacy | | STS | Security token service | | Secrets Manager | Secret storage | | Security Lake | Security data lake |

Monitoring & Logging

| Service | Description | |---------|-------------| | Amazon Managed Service for Prometheus | Managed Prometheus workspaces with data-plane passthrough | | CloudTrail | API audit logging | | CloudWatch | Metrics and alarms | | CloudWatch Logs | Log management | | X-Ray | Distributed tracing |

Networking & Content Delivery

| Service | Description | |---------|-------------| | API Gateway | API management (REST API) | | API Gateway v2 | API management (HTTP/WebSocket API) | | App Mesh | Service mesh | | CloudFront | CDN | | ELBv2 | Load balancing | | Global Accelerator | Network acceleration | | Location | Location-based services | | Route 53 | DNS service | | Route 53 Resolver | DNS resolver |

Application Integration

| Service | Description | |---------|-------------| | Amplify | Full-stack application hosting | | AppSync | GraphQL API | | Pinpoint SMS Voice v2 | SMS messaging | | SES | Email service | | SES v2 | Email service (v2 API) | | Scheduler | Task scheduling | | Step Functions | Workflow orchestration |

Management & Configuration

| Service | Description | |---------|-------------| | Backup | Centralized backup service | | Cloud Control API | Unified CRUD API for cloud resources | | CloudFormation | Infrastructure as code | | CodeConnections | Source code connections | | Config | Resource configuration | | Organizations | Multi-account management | | SSM | Systems Manager | | Service Quotas | Service limit management |

Analytics & ML

| Service | Description | |---------|-------------| | Athena | SQL query service | | Comprehend | NLP service | | Data Exchange | Data marketplace | | Entity Resolution | Entity matching | | Forecast | Time-series forecasting | | Glue | ETL service | | Rekognition | Image/video analysis | | SageMaker | Machine learning |

Developer Tools

| Service | Description | |---------|-------------| | CodeGuru Profiler | Application profiling | | CodeGuru Reviewer | Automated code review |

Other Services

| Service | Description | |---------|-------------| | Cost Explorer | Cost analysis | | DLM | Data lifecycle manager | | Directory Service | Microsoft AD | | EMR Serverless | Big data processing | | FinSpace | Financial data management | | GameLift | Game server hosting | | Resilience Hub | Application resilience |

Quick Start

Docker

docker run -p 4566:4566 ghcr.io/sivchari/kumo:latest

With data persistence:

docker run -p 4566:4566 \
  -e KUMO_DATA_DIR=/data \
  -v kumo-data:/data \
  ghcr.io/sivchari/kumo:latest

Binary

# Build
make build

Run

./bin/kumo

Run with data persistence

KUMO_DATA_DIR=./data ./bin/kumo

Docker Compose

services:
  kumo:
    image: ghcr.io/sivchari/kumo:latest
    ports:
      - "4566:4566"

With data persistence:

services:
  kumo:
    image: ghcr.io/sivchari/kumo:latest
    ports:
      - "4566:4566"
    environment:
      - KUMO_DATA_DIR=/data
    volumes:
      - kumo-data:/data

volumes: kumo-data:

Usage Examples

S3

package main

import ( "context" "strings"

"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/credentials" "github.com/aws/aws-sdk-go-v2/service/s3" )

func main() { cfg, _ := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"), config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider("test", "test", "")), )

client := s3.NewFromConfig(cfg, func(o *s3.Options) { o.BaseEndpoint = aws.String("http://localhost:4566") o.UsePathStyle = true })

// Create bucket client.CreateBucket(context.TODO(), &s3.CreateBucketInput{ Bucket: aws.String("my-bucket"), })

// Put object client.PutObject(context.TODO(), &s3.PutObjectInput{ Bucket: aws.String("my-bucket"), Key: aws.String("hello.txt"), Body: strings.NewReader("Hello, World!"), }) }

SQS

package main

import ( "context" "fmt"

"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/credentials" "github.com/aws/aws-sdk-go-v2/service/sqs" )

func main() { cfg, _ := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"), config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider("test", "test", "")), )

client := sqs.NewFromConfig(cfg, func(o *sqs.Options) { o.BaseEndpoint = aws.String("http://localhost:4566") })

// Create queue result, _ := client.CreateQueue(context.TODO(), &sqs.CreateQueueInput{ QueueName: aws.String("my-queue"), })

// Send message client.SendMessage(context.TODO(), &sqs.SendMessageInput{ QueueUrl: result.QueueUrl, MessageBody: aws.String("Hello from SQS!"), })

// Receive message messages, _ := client.ReceiveMessage(context.TODO(), &sqs.ReceiveMessageInput{ QueueUrl: result.QueueUrl, })

for _, msg := range messages.Messages { fmt.Println(*msg.Body) } }

DynamoDB

package main

import ( "context"

"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/credentials" "github.com/aws/aws-sdk-go-v2/service/dynamodb" "github.com/aws/aws-sdk-go-v2/service/dynamodb/types" )

func main() { cfg, _ := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"), config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider("test", "test", "")), )

client := dynamodb.NewFromConfig(cfg, func(o *dynamodb.Options) { o.BaseEndpoint = aws.String("http://localhost:4566") })

// Create table client.CreateTable(context.TODO(), &dynamodb.CreateTableInput{ TableName: aws.String("users"), KeySchema: []types.KeySchemaElement{ {AttributeName: aws.String("id"), KeyType: types.KeyTypeHash}, }, AttributeDefinitions: []types.AttributeDefinition{ {AttributeName: aws.String("id"), AttributeType: types.ScalarAttributeTypeS}, }, BillingMode: types.BillingModePayPerRequest, })

// Put item client.PutItem(context.TODO(), &dynamodb.PutItemInput{ TableName: aws.String("users"), Item: map[string]types.AttributeValue{ "id": &types.AttributeValueMemberS{Value: "user-1"}, "name": &types.AttributeValueMemberS{Value: "Alice"}, }, }) }

Secrets Manager

package main

import ( "context" "fmt"

"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/credentials" "github.com/aws/aws-sdk-go-v2/service/secretsmanager" )

func main() { cfg, _ := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"), config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider("test", "test", "")), )

client := secretsmanager.NewFromConfig(cfg, func(o *secretsmanager.Options) { o.BaseEndpoint = aws.String("http://localhost:4566") })

// Create secret client.CreateSecret(context.TODO(), &secretsmanager.CreateSecretInput{ Name: aws.String("my-secret"), SecretString: aws.String({"username":"admin","password":"secret123"}), })

// Get secret result, _ := client.GetSecretValue(context.TODO(), &secretsmanager.GetSecretValueInput{ SecretId: aws.String("my-secret"), })

fmt.Println(*result.SecretString) }

Configuration

Environment variables:

| Variable | Default | Description | |----------|---------|-------------| | KUMO_HOST | 0.0.0.0 | Server bind address | | KUMO_PORT | 4566 | Server port | | KUMO_LOG_LEVEL | info | Log level (debug, info, warn, error) | | KUMO_DATA_DIR | (unset) | Directory for persistent storage. When unset, data is in-memory only. |

Logging

kumo logs all requests with structured fields. The log level controls the amount of detail:

INFO (default)

Each request is logged with method, path, status, duration, and API action name:

level=INFO msg=request method=POST path=/ status=200 duration=61µs request_id=... target=secretsmanager.CreateSecret
level=INFO msg=request method=PUT path=/my-bucket pattern=/{bucket} status=200 duration=30µs request_id=...
  • target -- appears for JSON/Query protocol services (Secrets Manager, DynamoDB, SQS, etc.)
  • action -- appears for Query protocol services (EC2, SNS, etc.) when Action is in the URL query string

DEBUG

In addition to INFO output, the full request body is logged:

level=DEBUG msg="request body" request_id=... body={"Name":"my-secret","SecretString":"..."}

Enable with:

KUMO_LOG_LEVEL=debug ./bin/kumo

Data Persistence

By default kumo runs as a pure in-memory emulator -- all data is lost when the process stops. This is ideal for CI/CD pipelines where each test run starts from a clean state.

For local development, set KUMO_DATA_DIR to enable persistent storage:

KUMO_DATA_DIR=./data ./bin/kumo

When enabled:

  • On startup, each service loads its previous state from $KUMO_DATA_DIR/{service}.json.
  • On graceful shutdown (SIGTERM/SIGINT), each service saves its current state.
  • The data directory is created automatically if it does not exist.
  • Writes are atomic (tmp file + rename) to prevent corruption on crash.
  • Ephemeral state (SQS in-flight messages, S3 multipart uploads) is not persisted.
$KUMO_DATA_DIR/
  s3.json
  sqs.json
  dynamodb.json
  iam.json
  ...

kumo-specific Endpoints

kumo provides additional endpoints under the /kumo/ prefix for testing purposes. These are not part of any AWS API but are useful for verifying application behavior in tests.

| Method | Path | Description | |--------|------|-------------| | GET | /kumo/ses/v2/sent-emails | Retrieve a list of emails sent via the SES v2 SendEmail API | | GET | /kumo/pinpointsmsvoicev2/sent-messages | Retrieve a list of SMS messages sent via the Pinpoint SMS Voice v2 SendTextMessage API |

Example: Retrieving sent emails

curl http://localhost:4566/kumo/ses/v2/sent-emails

Response:

{
  "SentEmails": [
    {
      "MessageId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "FromEmailAddress": "[email protected]",
      "Destination": {
        "ToAddresses": ["[email protected]"]
      },
      "Subject": "Hello",
      "Body": "Hello, World!",
      "SentAt": "2025-01-01T00:00:00Z"
    }
  ]
}

Example: Retrieving sent SMS messages

curl http://localhost:4566/kumo/pinpointsmsvoicev2/sent-messages

Response:

{
  "SentTextMessages": [
    {
      "MessageId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "DestinationPhoneNumber": "+1234567890",
      "OriginationIdentity": "+0987654321",
      "MessageBody": "Hello!",
      "SentAt": "2025-01-01T00:00:00Z"
    }
  ]
}

Development

# Run tests
make test

Run integration tests

make test-integration

Lint

make lint

Build

make build

Contributing

Contributions are welcome! Please see the issues for planned features and improvements.

License

MIT License

Chat with me