An energy trading firm receives tens of thousands of price updates per second from several exchanges. Updates for any single contract must be consumed strictly in arrival order, the pipeline must survive a full regional outage with minimal data loss, and every raw update must be retained for seven years for audits. Which design best satisfies these needs?
Show answer and explanation
Answer: A
Amazon MSK provides Apache Kafka, where records within one partition keep their order, so keying partitions by contract preserves per-contract sequencing at high throughput. MirrorMaker 2 asynchronously replicates topics to a second region, which limits data loss during a regional failure. A sink connector writing to S3 with Object Lock in compliance mode meets the seven-year immutable retention requirement.
Why the other options are wrong
- B. Standard SQS queues do not guarantee ordering and can deliver messages more than once, so updates for one contract could be processed out of sequence. FIFO queues would be needed for ordering, and they still do not act as a replayable audit log.
- C. Firehose buffers records into batches and delivers them to a destination without preserving per-contract ordering for consumers. It is a delivery pipeline, not an ordered, replayable stream, and a single-region design fails the regional recovery need.
- D. Redis sorted sets live primarily in memory and are not designed as a durable seven-year audit archive. Snapshots exported to Glacier are periodic, so they cannot meet the goal of minimal loss and do not give ordered replay per contract.