This is the biggest domain at 28% of the scored exam. Expect scenario questions: “which customisation approach is cheapest for this need”, “what does lowering temperature do”, “which prompt technique”, “which metric evaluates summaries”, “where do embeddings live”. Learn the ladder of customisation, the prompt techniques, and the metric names.
Selecting a pre-trained model
| Criterion | What to ask |
|---|---|
| Cost | Price per token or per image; provisioned throughput needs |
| Modality | Text, image, audio, video, embeddings, multimodal input |
| Latency | Real-time chat needs a fast, smaller model |
| Multilingual | Does it support the languages you need? |
| Model size and complexity | Larger is usually more capable, slower and costlier |
| Customisation | Can you fine-tune or continue pre-training it? |
| Input and output length | Context window and maximum output tokens |
| Compliance and licensing | Region availability, terms, data handling |
Test several models on your own prompts. A smaller model that meets the quality bar is the right answer when cost or latency matters.
Inference parameters
These settings shape the response without changing the model.
| Parameter | Effect |
|---|---|
| Temperature | Randomness. Low (near 0) is focused, predictable and repeatable: facts, extraction, code. High is more varied and creative: brainstorming, stories |
| Top-p (nucleus sampling) | Considers only the smallest set of likely tokens whose combined probability reaches p. Lower is more conservative |
| Top-k | Considers only the k most likely next tokens. Lower is more conservative |
| Maximum tokens / response length | Caps output size; controls cost and cut-off |
| Stop sequences | Strings that make the model stop |
| Input length | Limited by the context window; long inputs cost more |
Temperature, top-p and top-k change randomness. They don’t add knowledge or fix hallucinations.
Retrieval Augmented Generation (RAG)
RAG gives the model relevant information at question time instead of retraining it. The model answers from retrieved passages placed in the prompt.
How it works
- Ingest: split documents into chunks, create an embedding of each with an embedding model, and store the vectors in a vector database (with the text and metadata).
- Retrieve: embed the user’s question, find the nearest vectors (semantic similarity search).
- Augment: add the retrieved passages to the prompt.
- Generate: the FM answers using that context, often with source citations.
Why it is popular
- Brings in current and private data without retraining.
- Reduces hallucinations by grounding answers in sources.
- Data stays in your control and can be updated by re-indexing.
- Cheaper and faster than fine-tuning.
- Permissions can be enforced on retrieval.
Business uses: customer support bots, internal knowledge assistants, policy and legal research, product Q&A, document search.
Amazon Bedrock Knowledge Bases
A fully managed RAG capability. You point it at a data source (S3, web crawler, Confluence, SharePoint, Salesforce and others); it chunks, embeds, stores and retrieves. You choose the embedding model and vector store. APIs include Retrieve (just passages) and RetrieveAndGenerate (full answer with citations). It can also query structured data stores by generating SQL.
AWS vector store options
| Service | Notes |
|---|---|
| Amazon OpenSearch Service (and OpenSearch Serverless) | Vector search engine; common default |
| Amazon Aurora PostgreSQL (and RDS for PostgreSQL) | pgvector extension; good when data is already relational |
| Amazon Neptune | Graph database; vector search plus graph relationships (GraphRAG) |
| Amazon DocumentDB (MongoDB compatibility) | Vector search on document data |
| Amazon MemoryDB / ElastiCache | In-memory vector search for very low latency |
| Amazon S3 Vectors | Low-cost vector storage in S3 |
| Third-party (Pinecone, Redis Enterprise Cloud, MongoDB Atlas) | Supported by Knowledge Bases |
The exam names OpenSearch, Aurora, Neptune, DocumentDB and RDS for PostgreSQL. A question that says “the team already uses PostgreSQL and wants to add vector search” points to Aurora or RDS for PostgreSQL with pgvector.
RAG versus Amazon Kendra
Kendra is an intelligent enterprise search service with connectors and natural-language queries; it can also act as a retriever for RAG. Knowledge Bases is the usual Bedrock-native RAG answer.
Agents for Amazon Bedrock
An agent lets an FM handle multi-step tasks: it reasons about a goal, breaks it into steps, calls action groups (APIs backed by Lambda functions, defined by an OpenAPI schema or function details), queries knowledge bases, and keeps conversation memory. Example: “rebook my flight” means looking up the booking, checking availability, calling the change API, and confirming. Bedrock AgentCore provides managed infrastructure to run and secure agents at scale, and Bedrock Flows chain prompts, models and services in a visual workflow. Agents can be wrapped with Guardrails, and you can trace each reasoning step.
Customising a foundation model: cost tradeoffs
From cheapest and simplest to most expensive:
| Approach | What changes | Cost and effort | Use when |
|---|---|---|---|
| Prompt engineering / in-context learning | Only the prompt (instructions, examples) | Lowest; no training | Start here always |
| RAG | Adds external knowledge at query time | Low to moderate; needs a vector store and ingestion | Answers need current or private data |
| Fine-tuning | Updates the model’s weights with labeled examples | High: training compute, data prep, hosting a custom model (provisioned throughput in Bedrock) | Specialised style, format or task behaviour |
| Continued pre-training | Further unsupervised training on a large domain corpus | Higher | The model lacks domain vocabulary and knowledge |
| Pre-training from scratch | Train an entire new model | Highest by far (millions, huge data) | Almost never |
Distillation (training a small model to mimic a large one) is another way to cut inference cost.
Mnemonic: prompt, RAG, fine-tune, continue pre-training, pre-train. Choose the lowest rung that meets the need.
Prompt engineering
Constructs
| Element | Purpose |
|---|---|
| Instruction | The task to do |
| Context | Background information, retrieved passages |
| Input data | What to process |
| Output indicator / format | The shape of the answer (JSON, bullets, length) |
| Persona / role | “You are a tax advisor” |
| Negative prompt | What to avoid or exclude (common in image generation, and for text: “do not mention pricing”) |
| Model latent space | The model’s internal learned representation; prompts steer which regions of what it learned the model draws on |
Techniques
| Technique | What you do |
|---|---|
| Zero-shot | Ask with no examples |
| Single-shot (one-shot) | Give one example |
| Few-shot | Give several examples so the model copies the pattern and format |
| Chain-of-thought (CoT) | Ask for step-by-step reasoning (“think step by step”); improves maths and logic |
| Prompt template | A reusable prompt with placeholders for variable input; gives consistency; Bedrock Prompt Management stores them |
| Role prompting | Assign a persona |
| Self-consistency / ReAct | Sample several reasoning paths, or alternate reasoning with tool actions |
Best practices
Be specific and concise; state the format; give examples; put instructions clearly apart from data (delimiters); break complex tasks into steps; experiment and iterate and keep what works; set guardrails for safety. Longer and vaguer prompts usually perform worse than clear ones.
Risks and limits
| Risk | Meaning |
|---|---|
| Prompt injection / hijacking | Malicious text in the input or retrieved data overrides instructions and redirects the model |
| Jailbreaking | Prompts crafted to get around the model’s safety rules |
| Poisoning | Harmful data planted in training data or retrieved documents so the model misbehaves |
| Exposure | Prompts or outputs leak sensitive data or the system prompt |
| Hallucination and inconsistency | Output varies and can be wrong |
Defences: input and output filtering (Bedrock Guardrails), least-privilege for agent tools, treat retrieved content as untrusted, human review for high-impact actions.
Training and fine-tuning
Training stages
- Pre-training: self-supervised learning on massive unlabeled data (predict the next token). Produces the general-purpose FM.
- Fine-tuning: supervised training on a smaller labeled dataset for a task or domain.
- Continued pre-training: keeps training on unlabeled domain data (medical, legal) to add knowledge, using the same objective as pre-training.
Fine-tuning methods
| Method | Meaning |
|---|---|
| Instruction tuning | Train on prompt and ideal response pairs so the model follows instructions better |
| Domain adaptation | Fine-tune on industry-specific data |
| Transfer learning | Reuse a model trained on one task as the starting point for another; the general idea behind all fine-tuning |
| Parameter-efficient fine-tuning (PEFT, for example LoRA) | Update only a small number of added parameters; cheaper and faster |
| RLHF | Humans rank outputs, a reward model is trained from those rankings, and the model is optimised to produce preferred responses (more helpful, less harmful) |
Bedrock offers fine-tuning (labeled prompt and completion pairs in S3) and continued pre-training (unlabeled data) for supported models; the result is a private custom model served with provisioned throughput.
Preparing fine-tuning data
- Data curation: clean, deduplicate, remove errors and harmful content.
- Governance: you have rights to the data; privacy and PII handled; lineage recorded.
- Size: enough examples (hundreds to thousands for fine-tuning; far more for continued pre-training); quality beats quantity.
- Labeling: accurate, consistent labels or ideal responses (SageMaker Ground Truth, human annotators).
- Representativeness: covers the real range of inputs and groups the model will see.
- RLHF data: human preference rankings.
Common failure: catastrophic forgetting, where fine-tuning erodes general abilities.
Evaluating foundation models
Approaches
| Approach | Notes |
|---|---|
| Human evaluation | People rate helpfulness, accuracy, tone. Slow and costly but captures what metrics miss |
| Benchmark datasets | Standard test sets (for example GLUE, SuperGLUE, MMLU, HELM) compare models on common tasks |
| Automatic metrics | Computed scores against reference answers |
| LLM-as-a-judge | Another model grades outputs |
Amazon Bedrock model evaluation supports automatic, human (your own team or an AWS-managed team) and LLM-as-a-judge evaluation, including RAG evaluation. SageMaker Clarify and FMEval also evaluate FMs.
Metrics
| Metric | Measures | Typical task |
|---|---|---|
| ROUGE (Recall-Oriented Understudy for Gisting Evaluation) | Overlap of n-grams and sequences between generated and reference text, recall-oriented | Summarisation |
| BLEU (Bilingual Evaluation Understudy) | n-gram precision against reference translations | Machine translation |
| BERTScore | Semantic similarity using embeddings, so paraphrases score well | Text generation where wording differs |
| Perplexity | How well the model predicts text; lower is better | Language modelling |
| F1, accuracy | Classification tasks | Q&A with exact answers, classification |
Rule: ROUGE equals summaries; BLEU equals translation; BERTScore equals meaning rather than exact words.
Does it meet business goals?
Tie the model to outcomes: productivity (time saved), user engagement (usage, satisfaction, retention), and task completion or quality on the real job. Run an A/B test or pilot. Cost per task and latency matter as much as scores.
Common traps
- A question needs up-to-date company data, and an option says fine-tune. Choose RAG.
- Fine-tuned custom models in Bedrock need provisioned throughput.
- “Make the output more deterministic” means lower temperature.
- ROUGE is for summarisation, BLEU for translation.
- Prompt engineering needs no training and is always the first thing to try.
- Embeddings stored in a vector database are the retrieval index; they are not the generator.
Exam checklist
- List selection criteria for a model.
- State what temperature, top-p, top-k and max tokens do.
- Describe the four RAG steps and name five AWS vector store options.
- Explain what a Bedrock agent does and what an action group is.
- Order customisation approaches by cost.
- Distinguish zero-shot, one-shot, few-shot and chain-of-thought; name four prompt risks.
- Distinguish pre-training, fine-tuning, continued pre-training, instruction tuning and RLHF.
- Match ROUGE, BLEU and BERTScore to tasks.