A Dataflow streaming job enriches events with location data from a third-party lookup API that enforces a request quota. Which design keeps the job within that quota while keeping throughput high?
Show answer and explanation
Answer: A
The GroupIntoBatches transform groups elements into bounded batches, so the job can send one request for many events rather than one per event. Pairing batching with a client-side rate limiter inside the DoFn keeps the outgoing request rate under the quota, even when the stream spikes. Together these reduce request volume and protect the external service. Per-event calls exceed quotas at peak, polling a side input creates constant unnecessary traffic, and unbounded state keeps growing and becomes costly to store and read.
Why the other options are wrong
- B. One request per element multiplies traffic with the event rate, so spikes will hit the quota and cause throttling errors for the job.
- C. Polling a side input continuously sends unnecessary requests to the API and does not bound the request rate for per-event enrichment lookups.
- D. Unbounded per-key state keeps growing as new keys arrive, and it does not enforce the request limit on first-time lookups, so the quota can still be exceeded.