← Back to artemrivnyi.com
SRE & Observability Case Study

AWS S3 Simulator Architecture: Slashing Cloud Egress Costs with S3-Compatible Emulation & Observability

By Artem Rivnyi • SRE & Cloud Infrastructure • October 2026 • Reference: aws-s3-simulator ↗
Status: Production Live
Cold Boot SLA: <1.4s
Memory Footprint: <85MB RAM
API Compliance: AWS S3 REST v4
Cloud Bill: $0.00 Egress

In modern cloud architectures, microservices interact constantly with object storage—uploading user avatars, writing database backups, archiving analytical event logs, and serving static assets. However, pointing automated test suites or local development environments directly at live AWS Simple Storage Service (S3) introduces substantial operational friction.

Testing against production cloud buckets causes unnecessary egress data transfer bills, increases CI/CD pipeline runtimes due to internet roundtrips, risks accidental modification of production assets, and requires provisioning long-lived AWS IAM secret keys on developer workstations. AWS S3 Simulator was designed to eliminate these bottlenecks by providing a high-fidelity, self-contained S3-compatible microservice with full Grafana observability.

1. Architectural Blueprint

The system packages an enterprise S3-compatible object storage core with a management REST API and a full Prometheus/Grafana observability stack:

Isolated S3 Emulation & Observability Harness Sub-1.4s Cold Boot • Zero Egress Cost
S3 Telemetry: Standby • Click "Simulate Live Flow" to trace object storage ingestion
Client Test Suite / AWS SDK (boto3)
S3 REST API v4
MinIO S3 Engine & Flask Management API
Docker Single Container • <85MB RAM
Prometheus Metrics Telemetry

Captures throughput (RPS), status code splits (2xx/4xx/5xx), and millisecond latency histograms.

Loki & Promtail Structured Logs

Real-time log ingestion directly from container stream without third-party forwarders.

Autoprovisioned Grafana Dashboard
Dashboard as Code

2. Core Engineering Challenges & Solutions

A. Achieving 100% AWS SDK API Compatibility

A mock service is useless if application developers have to rewrite their codebase when deploying to real AWS. By embedding the MinIO S3 Engine at its core, the simulator fully supports the standard AWS S3 REST API specification (v4 signature authentication, multipart uploads, bucket lifecycle policies, and presigned URLs). Standard AWS SDKs (`boto3` for Python, `@aws-sdk/client-s3` for Node.js, `aws-sdk-go`) function seamlessly simply by overriding the endpoint_url parameter.

B. Full-Stack SRE Observability (Prometheus + Loki + Grafana)

Rather than treating local storage as a black box, the architecture integrates a complete production-grade observability harness:

C. Container Footprint & Startup Optimization

To serve seamlessly inside fast CI/CD runners (GitHub Actions, GitLab CI), the entire simulator runtime is compiled into a multi-stage Docker image that boots and passes health checks in under 1.4 seconds, consuming less than 85 MB of idle memory.

3. Performance & Cost Impact

Metric Live AWS S3 in CI/CD AWS S3 Simulator (Local) Efficiency Gain
Upload Latency (10 MB payload) 480 – 1,200 ms 14 – 28 ms ~97% faster transfer
CI Test Suite Execution 3 min 45 sec 1 min 12 sec 68% pipeline speedup
Cloud Data Transfer Egress Cost $0.09 / GB egress $0.00 (Zero cloud egress) 100% cost elimination
IAM Credential Leak Risk Moderate (AWS secrets in CI) Zero (Local ephemeral credentials) Zero blast radius

4. Key Production Lessons

  1. Shift Cloud Dependencies Left: Emulating cloud services locally shortens developer feedback loops from minutes to milliseconds and eliminates friction during offline development.
  2. Observability is Not Just for Production: Having Grafana dashboards running against local integration tests allows engineers to detect N+1 storage query antipatterns before code ever merges to main.
  3. Ephemeral Test State Prevents Flaky Tests: Running storage in isolated Docker containers ensures each integration test run starts from a completely pristine filesystem state.
TECHNICAL DEEP DIVES

Explore More Architecture Case Studies