This is the largest domain (31% of scored content) and it covers everything that happens before a user sees an answer: picking the model, preparing data, storing and retrieving context, and managing prompts. Questions are scenarios (“a company wants…”) with a constraint such as latency, regional availability, data freshness or governance. You pick the managed AWS feature that fits with the least custom code, and most of the traps are about retrieval (chunking, embeddings, metadata, hybrid search) and model access (Cross-Region inference, switching models without redeploying).
Designing the solution
Start from requirements: task type (chat, summarisation, extraction, code, agentic), latency target, volume, data sensitivity, languages, modalities, budget. Then choose the integration pattern.
| Pattern | Use when |
|---|---|
| Prompt only (zero or few-shot) | Task is general and the model already knows the domain |
| RAG | Answers must use private or fast-changing data, need citations, or must not be baked into model weights |
| Fine-tuning or continued pre-training | You need a consistent style, format or domain vocabulary that prompts can’t achieve; data is stable |
| Agent with tools | The model must take actions or call APIs across several steps |
| Workflow (Step Functions, Prompt Flows) | The steps are known in advance; you want determinism, retries and audit |
Rules of thumb that recur: RAG before fine-tuning for knowledge; fine-tuning for behaviour and format; a workflow beats an agent when the path is fixed.
Proof of concept. Validate feasibility cheaply on Amazon Bedrock before building: on-demand invocation, the Bedrock playground, a small knowledge base, and a representative test set. Measure quality, latency and token cost per request, then extrapolate. Don’t buy Provisioned Throughput or fine-tune during a PoC.
Standard components. Use the AWS Well-Architected Framework and the Generative AI Lens (in the Well-Architected Tool) to review designs. Build shared, reusable pieces (a prompt library, a gateway, guardrail configurations, IaC modules) so every team gets the same logging, security and cost controls.
Selecting foundation models
Evaluate on your own data, not on leaderboard scores alone. Criteria: task quality, context window, modality, latency, cost per million input and output tokens, language support, regional availability, customisation support, licensing and data-handling terms.
- Bedrock gives serverless access to models from Amazon (Nova, Titan), Anthropic, Meta, Mistral, Cohere, AI21, DeepSeek, OpenAI open-weight models and others through one API. Models are enabled for use by default in most Regions, subject to IAM and (for some third-party models) marketplace terms.
- SageMaker AI (JumpStart, your own containers) when you need a model Bedrock doesn’t offer, full control of the serving stack, or a fine-tuned open-weight model with custom hosting.
- Use Amazon Bedrock Model Evaluation (below, and in the testing domain) to compare candidates with the same dataset.
Flexible model access: switching without code changes
The exam likes “change the model or provider without redeploying”.
- Keep the model ID, inference parameters and prompt template in configuration: AWS AppConfig (with feature flags, gradual rollout and rollback), Parameter Store, or DynamoDB. The Lambda function reads the value at run time.
- Put an abstraction layer in front: API Gateway plus Lambda that normalises requests and routes to Bedrock, SageMaker or an external provider. Clients never see the model ID.
- The Bedrock Converse API presents one request and response shape across models that support it, which is what makes swapping cheap.
InvokeModeluses each model’s native body format, so code changes with the model. - AppConfig is the right answer when the question says gradual, rollback or validation of configuration; Parameter Store when it says simple or cheapest.
Resilient model access
| Technique | What it does |
|---|---|
| Cross-Region inference (inference profiles) | A system-defined profile routes requests across Regions in a geography (for example us. or eu. prefixed IDs) or global, to get higher throughput and handle bursts and models with limited regional availability. You call the profile ID or ARN instead of the model ID. Data stays within the chosen geography for geographic profiles |
| Application inference profiles | You create a profile for a model or Region set and tag it, so you can track cost and usage per application or tenant |
| Step Functions circuit breaker | Track failures in DynamoDB; when a threshold is hit, the state machine stops calling the failing model and routes to a fallback |
| Fallback model | Retry on a different model or Region when the primary throws a throttling or timeout error |
| Graceful degradation | Return a cached answer, a smaller model’s answer or a static message instead of an error |
| Retries with exponential backoff and jitter | The AWS SDK does this by default for throttling; tune it, don’t write your own loop |
Cross-Region inference is the answer to “throttling during peaks” and “model not available in my Region”. It is not a disaster recovery plan for your own data stores.
Customising and deploying models
Customisation options in Bedrock: fine-tuning (labelled prompt and completion pairs), continued pre-training (unlabelled domain text), model distillation (a large teacher model generates data to train a smaller student) and Custom Model Import (bring weights of supported open architectures). Fine-tuned Bedrock models are served through Provisioned Throughput.
On SageMaker AI:
- Parameter-efficient fine-tuning (LoRA and adapters) trains a small set of added weights, so it’s cheaper and faster than full fine-tuning and the adapter is small to store and swap. SageMaker inference components can host many adapters on one endpoint.
- SageMaker Model Registry versions models, stores metadata and approval status; an approval event can start a deployment pipeline (CodePipeline or Step Functions).
- Deploy with blue/green or canary endpoint updates and automatic rollback on CloudWatch alarms; retire old versions deliberately and update the registry status.
- Model cards document intended use, training data and limitations (covered under governance).
Data validation and processing for FMs
- Validation: AWS Glue Data Quality (DQDL rules for completeness, uniqueness, freshness), SageMaker Data Wrangler, custom Lambda checks, and CloudWatch metrics and alarms on failure counts. Reject or quarantine bad records (an S3 prefix or a dead-letter queue) before they reach the embedding step.
- Modalities: text through Lambda or Glue; images through multimodal models (Nova, Claude) or Titan Multimodal Embeddings; audio through Amazon Transcribe then text; documents and scans through Amazon Textract or Bedrock Data Automation (extracts structured data from documents, images, audio and video); tabular data through SageMaker Processing or Glue. Large batch preparation: SageMaker Processing jobs.
- Formatting input: each model has a request shape. Bedrock
InvokeModeltakes a JSON body in the model’s format; Converse takes amessagesarray of roles (user,assistant) with content blocks, plus a separatesystemfield. Dialog apps must resend history on every call because the model is stateless. SageMaker endpoints use the container’s expected payload. - Improving input quality: normalise whitespace and encodings with Lambda, extract entities with Amazon Comprehend, summarise or reformat messy text with a small Bedrock model, de-duplicate, and remove boilerplate (headers, footers, navigation).
Vector stores
A vector store holds embeddings plus text and metadata and supports nearest-neighbour search.
| Option | Notes |
|---|---|
| Bedrock Knowledge Bases (managed) | Handles ingestion, chunking, embedding, storage and retrieval. Quick-create uses OpenSearch Serverless; you can also choose a store you manage |
| OpenSearch Serverless / OpenSearch Service | k-NN vector search, hybrid (keyword plus vector), filtering, sharding control (managed domains). Neural plugin integrates with Bedrock embedding models |
| Aurora PostgreSQL with pgvector | Fits when data is already relational and you want SQL joins with vectors |
| Amazon S3 Vectors | Lowest cost for large, infrequently queried vector sets; native vector storage in S3 |
| Neptune Analytics | Graph plus vector (GraphRAG) |
| DynamoDB | Good for metadata and document records alongside a vector store, not itself a vector search engine |
| MemoryDB, Pinecone, Redis Enterprise Cloud, MongoDB Atlas | Supported as Knowledge Base stores or alternatives |
Organisation and scale. Use multiple indexes for distinct domains or tenants, hierarchical indexing (summary first, then detail), and sharding so large collections spread across nodes. Pick OpenSearch when you need hybrid search, aggregations and tuning at scale; pgvector when relational joins matter.
Metadata. Attach timestamps, author, domain, department, language and access labels. In Knowledge Bases, put a <file>.metadata.json next to each S3 document; S3 object metadata and tags are the other sources. Retrieval then supports metadata filters (equals, in, greater-than, and/or), which gives higher precision and a way to enforce per-user or per-tenant visibility.
Integrations. Knowledge Bases connect to S3, web crawler, Confluence, SharePoint and Salesforce; custom sources use direct ingestion APIs. Wikis and document systems without a connector are reached with Lambda and API-based sync into S3.
Keeping data current.
- Knowledge Bases: run sync (ingestion job) on a schedule (EventBridge Scheduler) or on events (S3 event to Lambda calling
StartIngestionJob). Syncs are incremental: only added, changed and deleted files are processed. - Custom pipelines: S3 events or DynamoDB Streams trigger Lambda to re-embed changed items and delete stale vectors; a scheduled full refresh catches drift.
- Delete vectors when the source is deleted, or answers will cite removed content.
Retrieval mechanisms
Chunking controls what a retrieved passage looks like.
| Strategy | Behaviour | Pick when |
|---|---|---|
| Default | About 300 tokens, sentence-aware | Quick start |
| Fixed-size | You set max tokens and overlap | Uniform text |
| Hierarchical | Small child chunks are searched; larger parent chunks are returned for context | Long structured documents; precise match plus broad context |
| Semantic | Splits where meaning changes, using an FM | Varied topics in one document; costs more |
| None | One chunk per file | Files already small and self-contained |
| Custom (Lambda) | Your own logic, also for metadata | Special formats, structure-aware splitting |
Too-small chunks lose context; too-large chunks dilute relevance and waste tokens. Overlap protects boundaries. For complex PDFs, enable an advanced parser (a foundation model or Bedrock Data Automation) so tables and layout survive.
Embeddings. The embedding model converts text to vectors; use the same model for indexing and querying and don’t mix models in one index. Titan Text Embeddings V2 supports 256, 512 or 1024 dimensions and long inputs (8,192 tokens); smaller dimensions save storage and speed at some accuracy cost. Titan Multimodal Embeddings handle image and text; Cohere Embed handles multilingual use. Changing the embedding model means re-embedding everything. For bulk embedding, batch requests in Lambda or use batch inference.
Search.
- Semantic (vector) search understands meaning but can miss exact terms such as part numbers, error codes or names.
- Hybrid search combines keyword (BM25) and vector results; Knowledge Bases support it on OpenSearch and some other stores. Choose it when queries contain identifiers.
- Reranking: retrieve more candidates (for example 25), then a reranker model (Amazon Rerank, Cohere Rerank) reorders them and you keep the best few. Raises relevance without re-indexing. Available through the Rerank API and as a Knowledge Base option.
- Query handling: query expansion (synonyms or a rewritten query from an FM), decomposition (split a multi-part question; the Knowledge Base option does this, or use Lambda or Step Functions), transformation (rewrite chat follow-ups into standalone queries using history).
APIs. Retrieve returns chunks and scores; RetrieveAndGenerate retrieves and produces a cited answer in one call (it manages session context). Use Retrieve when you need control over the prompt, reranking or a non-Bedrock generator.
Consistent access for models. Expose retrieval as a tool (function calling or Converse tool use) or through an MCP server so any agent or client queries the store the same way; Bedrock Agents can attach a knowledge base directly.
Prompt engineering and governance
Instruction frameworks. A system prompt sets role, rules, tone and output format; user input goes in a separate message. Techniques: few-shot examples, explicit output schemas, chain-of-thought (“reason step by step”, or the model’s extended thinking), delimiters (XML-style tags) to separate instructions from data, and negative instructions kept short. Guardrails enforce policy that prompts alone can’t guarantee.
Context and conversation. Models are stateless: store history in DynamoDB (session ID key, TTL to expire), trim or summarise old turns, and use Comprehend or an FM for intent detection. Step Functions can run a clarification loop (ask for missing slots before calling the model) and human-in-the-loop waits.
Bedrock Prompt Management. Stores parameterised templates (variables in {{double braces}}), versions (immutable snapshots), variants (compare prompts, models or parameters) and a draft. Applications reference a prompt by ARN and version, so prompts are updated without a code deploy. Add approval workflows around version creation, keep templates in S3 or source control, track use with CloudTrail, and log access to CloudWatch Logs.
Prompt quality assurance. Regression-test prompts against a fixed set of inputs on every change: Lambda checks expected format or keywords, Step Functions runs edge cases in parallel, CloudWatch charts pass rates. Gate prompt promotion on the results.
Prompt Flows. A visual builder for sequential chains, condition branching, iterator and collector loops, and nodes for prompts, knowledge bases, agents, Lambda, Lex and S3. Use it for no-code chains with pre- and post-processing and reusable components. For long-running, audited or error-handling-heavy logic, Step Functions is stronger.
Common traps
- Fine-tuning to add facts that change weekly: use RAG instead.
- Mixing embedding models or dimensions between ingestion and query.
- Fixed-size chunks splitting tables or code; use hierarchical, semantic or custom.
- Treating Cross-Region inference as changing data residency: geographic profiles stay in-geography, global ones don’t.
- Hard-coding the model ID in code when the question says “without redeploying”.
- Metadata filters can’t be applied if the metadata file isn’t synced; re-sync after adding it.
Exam checklist
- RAG vs fine-tuning vs agent vs workflow, and which fits which need.
- AppConfig or Parameter Store plus an API Gateway or Lambda abstraction for swapping models.
- Cross-Region inference profiles for throughput and availability; application profiles for cost tracking.
- Circuit breaker with Step Functions and DynamoDB; fallback and graceful degradation.
- LoRA and adapters, Model Registry, canary and rollback on SageMaker.
- Glue Data Quality, Transcribe, Textract, Bedrock Data Automation, Comprehend roles.
- Knowledge Base vector stores and when to pick OpenSearch, pgvector, S3 Vectors, Neptune Analytics.
- Chunking strategies; Titan V2 dimensions; same model at index and query time.
- Hybrid search, reranking, query decomposition, metadata filters.
- Retrieve vs RetrieveAndGenerate; retrieval exposed as a tool or MCP server.
- Prompt Management versions and variants; Prompt Flows nodes; Converse API system prompt and history.