Resilient cloud solutions

Key terms

• RPO (recovery point objective): how much data you can lose, such as 5 minutes. RTO (recovery time objective): how long until service is restored.
• High availability survives an Availability Zone failure (Multi-AZ). Disaster recovery survives a Region failure (multi-Region).
• Static stability: survive losing an AZ without having to launch anything. To keep capacity N after losing one of two AZs, run N in each (2N total); with three AZs, N/2 per AZ.

Compute

EC2

• A golden AMI is pre-baked with agents, patches and the app, so instances boot fast. User data runs once at first boot as root. Require IMDSv2 for instance metadata.
• Status checks: a system check failure is AWS hardware or network — fix with a CloudWatch alarm using the recover action (same instance ID, IPs and EBS on new hardware) or stop/start. An instance check failure is the OS — reboot. Supported types also recover automatically by default.
• Spot instances get a 2-minute interruption warning as an EventBridge event.

Auto Scaling groups

• Keep between min and max instances across AZs using a launch template (launch configurations are legacy), replacing unhealthy instances.
• Health check type: EC2 (default — only notices dead instances) or ELB (also replaces instances failing the load balancer’s health check). The grace period pauses checks during boot; if it’s shorter than start-up time, new instances are killed in a loop.

• Lifecycle hooks pause instances:
  • Pending:Wait on launch — install software, register.
  • Terminating:Wait before termination — copy logs to S3, deregister.
  • The hook emits an event (for example EC2 Instance-terminate Lifecycle Action) to EventBridge → Lambda or Run Command does the work → CompleteLifecycleAction. Otherwise it continues when the heartbeat timeout (default 1 hour) expires.
• Warm pools: pre-initialised stopped, running or hibernated instances that join quickly — fixes slow-booting apps without paying for full running capacity.
• Instance refresh: rolling replacement after a new AMI or launch template version, with minimum healthy percentage, checkpoints, alarms, auto rollback and skip-matching.
• Operations: suspend processes (HealthCheck, ReplaceUnhealthy, AZRebalance, Terminate) while troubleshooting; Standby takes an instance out of service; scale-in protection protects specific instances.
• Termination policy decides which instance goes first (default: balance AZs, then oldest launch template). Mixed instances policy blends On-Demand and Spot; capacity rebalancing replaces Spot instances at risk of interruption.
• Graceful termination: the target group’s deregistration delay drains in-flight requests; a Terminating:Wait hook does cleanup.

Containers

• ECS: task definition (image, CPU/memory, ports, environment, secrets, log driver) → task → service (keeps N tasks behind a load balancer) → cluster. EC2 or Fargate launch type.
• Two roles: the task execution role (ECS agent pulls from ECR, writes logs, reads secrets for injection) and the task role (what your code can do).
• Service Auto Scaling with target tracking on CPU, memory or request count; capacity providers scale the EC2 layer.
• Placement: strategies binpack, spread, random; constraints distinctInstance and memberOf. Dedicated hardware for one service = its own capacity provider backed by its own Auto Scaling group.
• EKS: managed node groups or Fargate; pods get AWS permissions through IRSA or EKS Pod Identity. GitOps with Flux or Argo CD reconciles the cluster to Git.

Lambda

• Up to 15 minutes, 128 MB–10 GB memory (CPU scales with memory). Zip with layers, or a container image.
• Versions are immutable; an alias (such as prod) points to a version and can split traffic — that’s what CodeDeploy shifts.
• Concurrency — an exam favourite:
  • Unreserved functions share the account pool (default 1,000 per Region).
  • Reserved concurrency guarantees and caps a function. It does not fix cold starts. Setting it to 0 throttles (disables) the function.
  • Provisioned concurrency keeps environments initialised — no cold starts. Set it on a version or alias and scale it with Application Auto Scaling (scheduled or target tracking) for daily patterns. SnapStart helps Java, Python and .NET.
• Invocation types: synchronous (API Gateway, ALB — caller handles errors); asynchronous (S3, SNS, EventBridge — 2 retries, then a DLQ or on-failure destination); event source mappings (SQS, Kinesis, DynamoDB Streams — batch size, batching window, partial batch response, bisect on error, max retries).
• VPC Lambdas need a NAT gateway or VPC endpoints to reach the internet and AWS APIs. Intermittent timeouts to a database usually mean cold starts plus connection setup: use RDS Proxy, provisioned concurrency, and open connections outside the handler.

API Gateway

REST, HTTP and WebSocket APIs. Deployments publish to stages; a stage canary sends a share of traffic to a new deployment. Throttling and usage plans protect backends.

Data stores

RDS and Aurora

• RDS Multi-AZ: synchronous standby in another AZ, automatic failover in 1–2 minutes, same endpoint. The standby isn’t readable — except in the Multi-AZ DB cluster variant with two readable standbys.
• Read replicas: asynchronous, readable, cross-Region, promotable (manual DR). Automated backups give point-in-time recovery up to 35 days; snapshots copy across Regions and accounts. RDS Proxy pools connections and speeds up failover.
• Aurora: six copies of data across three AZs; up to 15 replicas that double as failover targets (priority tiers). A single-instance cluster has no failover target — add a reader in another AZ.
• Aurora endpoints:

• Aurora Global Database: a primary Region plus secondary read-only Regions, storage-level replication typically under a second (RPO about 1 s), managed switchover and failover in about a minute. Also Serverless v2, Backtrack (MySQL — rewind without restoring) and cloning. Multi-master is retired.

DynamoDB

• Capacity: on-demand or provisioned with auto scaling. If WriteThrottleEvents rise, raise the auto scaling maximum or switch to on-demand. A throttled GSI throttles writes to the base table.
• Global tables: multi-Region, active-active; conflicts resolve as last writer wins. Streams (24 h change log), PITR (35 days), DAX (read cache), TTL.

S3 and other storage

• 11 nines durability; lifecycle rules; versioning (deletes add a delete marker), MFA Delete, Object Lock (compliance mode: nobody can delete; governance: privileged users can override) and legal hold.
• Replication (CRR and SRR) needs versioning on both buckets and an IAM role; works cross-account (owner override). It copies new objects only — use S3 Batch Replication for existing ones. Replication Time Control replicates 99.99% of objects within 15 minutes. Delete markers and KMS-encrypted objects need explicit configuration.
• ElastiCache (Redis OSS / Valkey) with Multi-AZ failover and Global Datastore. EFS is regional with replication to another Region.

Messaging, streaming and orchestration

• SQS: visibility timeout hides a message in flight — for Lambda consumers set it to at least 6× the function timeout. Redrive to a DLQ after maxReceiveCount. FIFO for order and exactly-once processing. Watch ApproximateNumberOfMessagesVisible (backlog) and ApproximateAgeOfOldestMessage (how far behind consumers are). Scale consumers on backlog per instance or task = visible messages ÷ running count.
• SNS: push pub/sub; fan-out to many SQS queues; filter policies; FIFO topics.
• Kinesis Data Streams: ordered shards (1 MB/s or 1,000 records/s in, 2 MB/s out each), 24 h to 365 days retention. If Lambda falls behind (IteratorAge grows), add shards or raise the parallelization factor (up to 10 batches per shard).
• Amazon Data Firehose: managed delivery to S3, OpenSearch, Redshift, Splunk or HTTP with buffering and optional Lambda transformation.
• Step Functions: state machines with Retry/Catch, parallel and map states. Standard (up to a year, exactly-once) or Express (high volume, short). Human approval with a .waitForTaskToken task.

Routing and failover

• Route 53 policies: simple, failover, weighted (canary, blue/green), latency, geolocation, geoproximity, multivalue, IP-based.
• Health checks: endpoint (from internet checkers), calculated (combine checks), or CloudWatch alarm-based — the way to health-check private resources the checkers can’t reach. Alias records can evaluate target health.
• Application Recovery Controller (ARC): readiness checks and highly reliable routing controls to flip traffic between Regions; zonal shift (and zonal autoshift) moves traffic away from an impaired AZ.
• CloudFront origin groups fail over between origins; Global Accelerator gives static anycast IPs with fast health-based Regional failover, including non-HTTP traffic.

Disaster recovery

• AWS Backup: central plans (schedule, retention, lifecycle) assigned by tags across EC2/EBS, RDS/Aurora, DynamoDB, EFS, FSx, S3 and more; cross-Region and cross-account copy; organization backup policies; Vault Lock in compliance mode for immutable backups; restore testing.
• Data Lifecycle Manager: EBS snapshot and AMI schedules by tag (for example, a daily AMI kept for 7 days). Use AWS Backup for multi-service, cross-account needs.
• AWS Elastic Disaster Recovery: continuous block-level replication of servers with fast failover — good for lift-and-shift DR.
• AWS Fault Injection Service: experiment templates with actions (stop or terminate instances, CPU and memory stress, network latency or blackhole, API throttling, AZ power interruption, ECS and EKS faults) against tagged targets, with stop conditions on CloudWatch alarms. Needs an IAM role.