A content team needs domain experts to rate generated answers for helpfulness and completeness on a custom scale. The ratings must come from people, not from a computed metric. Which approach should they use?
Show answer and explanation
Answer: D
Human evaluation jobs in Bedrock Model Evaluation let reviewers score responses against your own criteria, such as a Likert scale, a thumbs up or down, or a pairwise comparison. You can use internal staff or an AWS-managed workforce. Subjective dimensions like helpfulness are not captured reliably by automatic metrics. Guardrails filter content against policies rather than rating quality, and CloudWatch math summarizes numeric metrics such as token counts.
Why the other options are wrong
- A. Automatic metrics compare outputs against defined references. They cannot judge helpfulness the way a domain expert can.
- B. ApplyGuardrail checks content against policies such as denied topics. It does not produce quality ratings for answers.
- C. Token counts are numeric usage data. They say nothing about whether an answer is complete or helpful.