An ML team must run Llama 3.1 70B with its own inference container, a GPU instance type it chooses, and a vLLM serving configuration, because managed Bedrock hosting does not expose those controls. Which AWS service is the best fit?
Show answer and explanation
Answer: D
A SageMaker AI real-time endpoint lets the team choose the GPU instance type, package its own serving container with vLLM or another framework, and set scaling and endpoint configuration. That level of control is what managed Bedrock hosting does not expose. Custom Model Import removes server management but runs inside Bedrock's managed infrastructure with supported architectures. Q Business and Comprehend serve entirely different purposes.
Why the other options are wrong
- A. Q Business is an enterprise question-answering product for employees. It does not host open-source LLMs or expose inference infrastructure.
- B. Comprehend performs NLP analysis such as entity and sentiment detection. It is not a general LLM hosting platform.
- C. Custom Model Import brings weights into Bedrock but abstracts the serving stack, so the team cannot pick its own container and GPU type.