PMLE Monitoring AI solutions

A fintech retrains its risk model weekly. A candidate should replace the production model only when its ROC AUC beats the incumbent by a set margin and no subgroup error rate gets worse by more than a small tolerance. The whole decision must happen automatically inside the pipeline. What construct enforces this?

Show answer and explanation

Answer: A

Pipeline conditions gate downstream steps on runtime values. In Vertex AI Pipelines this is expressed with the Kubeflow DSL condition construct, which reads the evaluation outputs and lets the deployment step run only when both the overall gain and the subgroup check pass. Logging results without a gate still ships regressions. Running deployment in parallel removes the gate entirely, and a manual sign-off contradicts the automation requirement.

Why the other options are wrong

  • B. Logging metrics without a gate still lets a regressed model reach production, because nothing checks the values before deployment runs.
  • C. Running deployment alongside evaluation means the model ships before any check finishes, which defeats the purpose of having thresholds.
  • D. A manual sign-off can work, but it does not auto-promote and it adds delay, so it fails the stated automation requirement.

Keep going. 350 PMLE questions, answers explained after each one.

Practise monitoring AI solutions

Reviewed November 2026. Spotted a problem with this question? Report it from the practice tool.