Redesigning for Scale: The Architecture That Replaced a Single Point of Failure

When you inherit an 8-year-old system that just took down your live platform, you approach the redesign with a specific kind of urgency. Not panic. Just urgency. Every architectural decision carries the weight of knowing what the alternative looks like.

In the previous article, I walked through the production incident that forced this work. The goals were clear: eliminate the single point of failure, enable horizontal scaling, achieve zero-downtime deployments, and build in auto-healing. This article walks through the architecture we designed to meet those goals.

Portfolio reference: The complete Bidding Engine Redesign case study covers the full picture, architecture, results, costs, and engineering decisions.

Design Principles First

Before picking any specific technology, we agreed on constraints that would drive every decision:

Stateless containers. Each ECS task should hold no local state that matters beyond a single connection. Room memberships, presence data, and auction state must live in a shared layer, not in process memory. This is what makes horizontal scaling possible.

No single point of failure at any layer. The old architecture failed because one thing going down meant everything going down. The new architecture should tolerate failures at every layer independently: a container crash, a zone failure, even the Redis primary going down, without cascading into a full outage.

AWS-native where possible. Our team already runs on AWS. Choosing ECS over Kubernetes, ElastiCache over a self-managed Redis cluster, and ALB over Nginx reduces operational overhead without sacrificing capability.

Observable by design. Logging, health checks, and metrics from day one. Not bolted on later when something breaks.

The New Architecture

Here's the complete picture of what we built:

Let me walk through each layer.

Layer 1: Clients

Users connect from the web client (Vue.js) or mobile app (React Native). Both use the Socket.IO v4 client, which handles the WebSocket upgrade and automatic reconnection.

The v4 client upgrade was paired with the server upgrade. Socket.IO v4 has a substantially improved reconnection algorithm with exponential backoff and jitter, which dramatically reduces the thundering herd problem on server restarts. In the old setup, a process restart would cause every connected client to reconnect simultaneously, hammering the freshly started server. The jitter in v4 distributes those reconnection attempts over several seconds instead.

Layer 2: Application Load Balancer

The ALB does three things here:

• Sticky sessions via cookie-based routing, essential for WebSockets, explained in depth in Part 3
• Health checks every 30 seconds per target. Unhealthy tasks are deregistered automatically
• WebSocket protocol support. The ALB natively handles the HTTP to WebSocket upgrade, no Nginx needed

The sticky session configuration ensures that once a client's initial WebSocket handshake is routed to a specific Fargate task, all subsequent requests from that client go to the same task. Without this, reconnection attempts could land on a different task mid-session, which breaks the Socket.IO connection state.

Layer 3: ECS Fargate Cluster

Each Node.js Socket.IO server runs as a Docker container on ECS Fargate. Fargate means we're not managing EC2 nodes. AWS handles placement, scaling the underlying compute, and recovering from hardware failures. We define task definitions and Fargate handles the rest.

Here's the core of our task definition:

{
  "family": "rtc-service",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "containerDefinitions": [{
    "name": "rtc-service",
    "image": "<account>.dkr.ecr.ap-southeast-2.amazonaws.com/rtc-service:latest",
    "portMappings": [{ "containerPort": 3000, "protocol": "tcp" }],
    "secrets": [
      { "name": "REDIS_URL", "valueFrom": "arn:aws:ssm:ap-southeast-2:<account>:parameter/app/redis-url" }
    ],
    "healthCheck": {
      "command": ["CMD-SHELL", "node -e \"require('http').get('http://localhost:3000/health', r => process.exit(r.statusCode === 200 ? 0 : 1))\""],
      "interval": 10,
      "timeout": 5,
      "retries": 3,
      "startPeriod": 30
    },
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/rtc-service",
        "awslogs-region": "ap-southeast-2",
        "awslogs-stream-prefix": "rtc"
      }
    }
  }]
}

A few things worth noting: awsvpc network mode gives each Fargate task its own elastic network interface, so security group rules can target individual tasks. REDIS_URL comes from SSM Parameter Store rather than a hardcoded environment variable, keeping credentials out of the task definition. The health check runs an inline Node.js script that hits the /health endpoint directly, which is faster than installing curl in the image.

Auto-scaling policies watch two metrics:

• ALB active connection count, the most direct signal of real-time load
• CPU utilization, fallback for compute-heavy processing

When active connections exceed our configured high-water mark, ECS launches additional tasks. Each new task registers with the ALB and subscribes to Redis automatically. No manual intervention, no traffic disruption.

Multi-AZ deployment ensures tasks are spread across at least two Availability Zones. If an entire AZ has issues, the remaining tasks in healthy zones continue serving traffic without interruption.

Layer 4: Redis ElastiCache

This is the critical addition that makes the whole system work. Redis serves two roles:

Pub-sub coordinator. Using @socket.io/redis-adapter, every Socket.IO event emitted on any container is published to a Redis channel. All other containers subscribe to that channel and re-emit the event to their local connected clients. A bid placed by a user on Node 1 reaches every user on Node 2 and Node 3 within milliseconds, regardless of which container they're connected to.

Shared state store. Auction room memberships, viewer counts, and presence data live in Redis rather than in-process memory. Any container can read or write this state, and all containers see the same view.

ElastiCache is deployed Multi-AZ with automatic failover from primary to replica. If the Redis primary fails, failover completes in under 30 seconds and the adapter reconnects transparently. More on failure scenarios in Part 4.

Layer 5: E-Bidding Platform API

The backend auction business logic runs separately. It's a Node.js/TypeScript application connected to RDS PostgreSQL. The RTC layer is intentionally decoupled from the auction engine: it handles delivery of events, not processing of bids.

The API handles bid validation, price updates, and persistence. The RTC layer handles fan-out to connected clients. These two concerns stay separated.

The 5-Part Implementation

The architecture required five parallel work streams:

The rest of the series goes deep on each of these.

Tech Stack

Next in the series: Connecting a Fleet: How Redis Pub/Sub Makes Socket.IO Scale Across Containers →

Previous: The Day Our Auction Platform Went Dark ←