A team receives JSON responses from a vendor API. Each response contains nested objects and arrays of line items, and the team needs one row per line item in BigQuery at high volume. Which approach is best?
Show answer and explanation
Answer: B
Dataflow can read the vendor payloads in bulk, parse their nested structures, expand the arrays so each line item becomes its own record, and write the results to BigQuery. This runs in a scalable pipeline that handles high volume reliably. BigQuery can query JSON columns, but repeatedly parsing raw payloads in every query costs more and gives less consistent schemas than producing a flat table once. A function invoked per payload has limited throughput for bulk work. Storing text defers the problem and makes every analysis harder.
Why the other options are wrong
- A. Keeping raw JSON in one column pushes parsing into every query and forces analysts to repeat the same extraction logic. Queries become slower and more expensive than querying a flattened table.
- C. Per-payload function invocations write rows individually, which is slow and expensive at high volume and can exhaust quotas. It does not provide the bulk transformation the job needs.
- D. Text strings cannot be filtered, grouped or typed without parsing, so analysts would recreate the flattening work repeatedly. Storing text defers the problem rather than solving it.