AIP-C01 question patterns

AWS AppConfig (or Parameter Store) holding the model ID and parameters, read by Lambda behind an API abstraction.

Configuration changes take effect at run time; AppConfig adds gradual rollout and rollback.

Bedrock Knowledge Bases with RetrieveAndGenerate.

It manages ingestion, chunking, embedding, storage, retrieval and citations.

RAG with a knowledge base, synced on a schedule or on S3 events.

Fine-tuning bakes facts into weights and can't cite or update quickly.

Fine-tune (or distil) a model.

Behaviour and style are what customisation changes; knowledge is better served by retrieval.

Cross-Region inference profile.

It spreads requests across Regions in a geography for more throughput and availability.

Step Functions circuit breaker with failure counts in DynamoDB and a fallback model.

It stops calling the failing model and keeps the application answering.

SQS buffer with Lambda maximum concurrency, SDK backoff, quota increase.

Smooths bursts under the quota and retries throttles safely.

Provisioned Throughput.

Reserved model units give guaranteed capacity and are required for custom models.

Bedrock batch inference.

Asynchronous S3-to-S3 jobs at a lower price.

Application inference profiles with cost allocation tags.

Usage is attributed to the profile and shows in Cost Explorer.

Hybrid search (keyword plus vector).

Keyword matching handles exact tokens that embeddings blur.

A reranker model over a larger candidate set.

Reranking improves relevance without re-indexing.

Hierarchical chunking.

Small child chunks are matched; larger parent chunks are returned.

Semantic chunking.

An FM finds topic boundaries; costs more.

Metadata filters on retrieval (plus Lake Formation for data lake sources).

Filters restrict which chunks a caller can retrieve.

Incremental sync (StartIngestionJob) from a schedule or S3 event.

Only changed files are re-processed and stale vectors are removed.

Re-embed the whole index with the new model.

Index and query vectors must come from the same model.

Aurora PostgreSQL with pgvector.

Keeps vectors with the relational data.

Amazon S3 Vectors.

Native vector storage in S3 is cheapest for cold data.

Query decomposition (Knowledge Base option or Lambda or Step Functions).

Sub-queries retrieve focused context that is then combined.

Step Functions workflow rather than an autonomous agent.

Known steps are cheaper and more predictable as a state machine.

Step Functions task token (waitForTaskToken).

The workflow waits until the approver returns the token.

Iteration cap, timeouts, per-tool least-privilege roles, circuit breakers.

Bounds limit the damage from runaway behaviour.

MCP server on Lambda.

Pay-per-call serverless hosting with a standard protocol.

MCP server on ECS or Fargate.

Containers handle memory, connections and state Lambda can't.

AWS Agent Squad or a Bedrock supervisor agent.

Intent classification dispatches to the right agent with context.

Guardrails with the ApplyGuardrail API.

It evaluates text independently of the model.

Guardrails contextual grounding check with a threshold.

Scores groundedness and relevance against the source text.

Guardrails sensitive information filter (and Comprehend for pre-processing).

Detects PII types and blocks or masks them.

Amazon Macie.

It finds sensitive data in S3 objects.

Interface VPC endpoints for bedrock-runtime (and agent endpoints) with endpoint policies.

PrivateLink keeps traffic on the AWS network.

IAM condition on the guardrail identifier.

The policy denies calls made without it.

Text-to-SQL against a trusted database with a read-only role.

Results come from data, not model memory.

CloudTrail plus model invocation logging to encrypted S3 or CloudWatch Logs.

CloudTrail shows API calls; invocation logs hold prompts and responses.

SageMaker Model Cards.

They hold intended use, data and evaluation results.

Treat retrieved content as untrusted: delimit it, filter it, least-privilege tools.

Indirect prompt injection comes through data, not the user prompt.

Bedrock prompt caching with stable content first.

Cached prefix tokens cost less and return faster.

Semantic cache using embeddings and a similarity threshold.

Near-duplicate queries return stored answers.

Route to a small model and escalate complex ones (cascade or intelligent prompt routing).

Most queries don't need the large model.

Streaming with ConverseStream over WebSocket or server-sent events.

Time to first token drops, so it feels faster.

CloudWatch anomaly detection on token metrics and Cost Anomaly Detection.

Learned baselines catch bursts without fixed thresholds.

Enable model invocation logging and query with Logs Insights.

Logging is off by default.

Low temperature (and top-p or top-k limits).

Less randomness gives repeatable output.

Bedrock Model Evaluation with a custom dataset.

Evaluates quality across models with the same inputs.

LLM-as-a-judge with a rubric, calibrated against human ratings.

Programmatic metrics miss meaning; judges scale.

RAG evaluation: retrieval-only, then retrieve-and-generate.

Separates context relevance from answer faithfulness.

Regression suite as a pipeline quality gate plus canary rollout.

Scores below threshold block the release; canary limits exposure.

Check stopReason is max_tokens; raise limit or shorten format.

The generation hit the output cap.

Count tokens; chunk, summarise or retrieve less.

The request exceeded the context window.

Golden dataset run on a schedule with scored comparison.

Known answers expose regressions.