A multilingual support team is evaluating one language model for three jobs: answering product questions, condensing long tickets, and tagging customer mood. Which evaluation plan gives the most complete picture?
Show answer and explanation
Answer: D
Each job has its own quality dimension, so each needs a metric that measures it. Answer correctness is commonly scored with exact match or F1, summary overlap with ROUGE, and mood tagging with classification accuracy or F1 against labelled examples. Reporting them separately shows where the model is strong and where it fails.
Why the other options are wrong
- A. Averaging across tasks hides a failing task behind a strong one. A model that does well on mood tagging and poorly on summaries would look acceptable, which is misleading for release decisions.
- B. Skipping two of three jobs gives no evidence about them. Assuming similar performance is unverified and would miss regressions that matter to real users.
- C. Perplexity measures predictive fit to text, not whether answers are correct, summaries are faithful or moods are right. It is not a task-quality measure for these jobs.