An e-commerce platform writes daily order events as JSON files to S3. Analysts want the data converted to Parquet and organized into folders by date to reduce scan costs in Athena. Which approach is most suitable?
Show answer and explanation
Answer: D
A Glue ETL job reads the JSON into a DynamicFrame, which tolerates inconsistent records, and writes Parquet to S3 with the date column set as a partition key. Each date then lands under its own prefix, so Athena can skip irrelevant folders. A crawler only catalogs data, S3 Object Lambda transforms objects at retrieval time without storing a converted dataset, and DataBrew focuses on cleaning and masking rather than bulk format conversion.
Why the other options are wrong
- A. Custom classifiers help a crawler recognize formats while inferring schemas. They do not convert file formats or write partitioned output.
- B. Object Lambda transforms data as it is retrieved. It does not produce a stored, partitioned Parquet dataset that Athena can scan efficiently.
- C. DataBrew recipes suit cleaning and masking columns. The scenario needs bulk format conversion with partitioned output, which the ETL engine handles directly.