A logistics firm receives shipment events in one record type and carrier updates in another. The downstream sink accepts a single common record type, and the pipeline must combine both sources into one collection. Which steps achieve this?
Show answer and explanation
Answer: D
Flatten merges PCollections, but every input must share the same element type. A Map or ParDo step first converts each source into the shared record type, and Flatten then merges the converted collections into one output. This is the standard way to unify heterogeneous sources in Beam. CoGroupByKey and GroupByKey perform keyed joins or groupings rather than simple merges.
Why the other options are wrong
- A. CoGroupByKey joins the sources by key and produces grouped results per key. It does not simply place every element from both sources into one output stream.
- B. GroupByKey on each source gathers elements into groups, which changes their shape and is not needed to combine two streams into one.
- C. Union is not an Apache Beam transform. Beam's merging operation is Flatten, which requires the element types to be converted first.