Data ingestion and transformation
Practise only this domain. 68 questions with explanations, 20 at a time.
Start this domainSample questions
- An e-commerce platform writes daily order events as JSON files to S3. Analysts want the data converted to Parquet and organized into folders by date to reduce scan costs in Athena. Which approach is most suitable?Medium
- A data pipeline must wait for a new file to land in S3, start a Glue job, retry that job on failure, and send a notification when it completes. Which service coordinates these steps most suitably?Medium
- A team runs Apache Airflow on Amazon MWAA and wants DAG tasks to create and monitor S3 objects and start Glue jobs using AWS-aware operators. What should the team do?Medium
- Before loading incoming records into the warehouse, a team needs managed rule checks for completeness, uniqueness and value ranges on tables registered in the Data Catalog. Which AWS service provides these checks?Medium
- A platform team runs one Kinesis data stream with 12 shards that feeds a fraud model, a live dashboard, and an archive job. Each reader must get dedicated read bandwidth so that none of them slows the others. Under default quotas, how many registered consumers can use enhanced fan-out on this single stream?Hard
- A data engineer's AWS Glue job runs in us-east-1 and must transform Parquet files stored in an S3 bucket located in eu-west-1. Which statement describes the correct approach?Hard
- A nightly AWS Glue Spark job fails with ExecutorLostFailure after running for about 40 minutes. The driver logs show one executor's container being killed, and the failing stage processes a single partition that holds most of the data. Which cause is MOST likely?Hard
- A team creates a new AWS Glue batch ETL job from the console and leaves the timeout field at its default. After how long will Glue stop a run that is still executing?Easy
- An e-commerce company needs the orders table in Amazon RDS for PostgreSQL reflected in an Amazon Redshift warehouse within minutes, including updates and deletes made at the source after the initial load. Which approach fits best?Medium
- A Glue job reads JSON event files in which one field sometimes holds a string and sometimes a number, which breaks a fixed schema. Which Glue abstraction handles this per-record variation without failing the job?Medium
- A nightly pipeline writes curated sales tables. The team wants the Spark job itself to stop and prevent publication when a null-rate or uniqueness constraint fails. Which approach meets that need?Hard