A fraud-scoring pipeline ingests transaction batches whose columns change: vendors add new fields and retire old ones. The team wants new columns flagged for review rather than silently breaking training, while still tracking gradual value shifts. Which TFX components support this?
Show answer and explanation
Answer: A
TFX's data validation chain computes per-feature statistics with StatisticsGen, derives a schema with SchemaGen, and then ExampleValidator compares each new batch against that schema. Added or missing features and shifted distributions are reported as anomalies, so reviewers can accept expected evolution by updating the schema instead of letting bad inputs reach training silently.
Why the other options are wrong
- B. A hard-coded schema makes every vendor change a failure. The pipeline stops, which protects training but does not handle the expected evolution of new and retired fields, and it gives no drift visibility.
- C. Auto-detection at every load accepts whatever shape arrives, so a dropped or renamed column is absorbed silently. There is no statistical baseline to flag shifts, which is the opposite of the review the team wants.
- D. Copying the last schema skips the comparison step entirely, so new fields are never noticed and distribution changes go unmeasured. It preserves the old contract rather than detecting changes against it.