This domain is about 20% of the exam. Questions give a symptom (wrong facts, outdated answers, off-brand tone, inconsistent output) and ask for the right fix. The skill is knowing which lever addresses which problem: prompting, grounding or RAG, fine-tuning, sampling settings, or human review. A frequent wrong answer is “use a bigger model”.
Limitations of foundation models
| Limitation | What it is | Example |
|---|---|---|
| Data dependency | A model reflects the data it was trained on. Missing, poor or narrow data means weak results | A model with little data on a niche industry answers poorly |
| Knowledge cutoff | The model knows nothing after its training data ended and nothing about your private data | Asked about last week’s price change, it can’t know |
| Bias | Patterns in training data (stereotypes, under-represented groups) show up in output | Résumé summaries that favour one group |
| Fairness | Outcomes may treat groups unequally even without intent | Loan wording that differs by demographic |
| Hallucination | The model produces fluent, confident text that is false or invented, including fake citations | A made-up legal case or product feature |
| Edge cases | Unusual inputs outside what the model saw often | Rare slang, odd formats, unusual combinations |
Why hallucinations happen: an LLM predicts likely next tokens. It is built to sound right, not to check facts. It has no built-in source of truth.
Other limits worth knowing: limited context window, inconsistent output between runs, sensitivity to prompt wording, cost and latency for large models, and no true understanding or accountability.
Practices to overcome limitations
| Problem | Best fit | Why |
|---|---|---|
| Wrong or invented facts | Grounding / RAG | Gives the model real source text to answer from, with citations |
| Outdated knowledge | Grounding with Google Search or fresh enterprise data | Retrieves current information at the time of the question |
| No knowledge of private company data | RAG over your data | The model reads your documents without being retrained |
| Unclear, rambling or wrongly formatted output | Prompt engineering | Better instructions, role, format and examples |
| Needs a specialised tone, style, terminology or task the model does poorly | Fine-tuning (or prompt tuning) | Teaches the model from your labeled examples |
| High-stakes decisions, legal or medical content | Human in the loop (HITL) | A person reviews and approves before action |
| Unsafe or off-topic output | Safety settings and filters, plus guardrails | Blocks categories of harmful content |
| Quality drifts over time | Monitoring and evaluation | Detects regression and triggers action |
Order of effort
Try the cheapest lever first.
- Prompt engineering: free and instant.
- Grounding / RAG: moderate effort, solves facts and freshness.
- Fine-tuning: needs quality labeled data, time and cost. Use when style or specialised task behaviour is the gap.
- Training from scratch: rarely justified.
Key distinction: fine-tuning teaches a model new behaviour or style; it is a poor way to add facts that change. For changing facts, use RAG. RAG is cheaper to update (change the documents, not the model) and can cite sources.
Human in the loop
Put a person in the workflow where mistakes are costly: approving outputs, labelling data, reviewing edge cases, giving feedback that improves the model. It adds accountability. The trade-off is speed and cost, so apply it by risk level.
Continuous monitoring and evaluation
Launching is not the finish line. Google recommends:
| Practice | Meaning |
|---|---|
| Key performance indicators (KPIs) | Define measurable goals up front (accuracy, resolution rate, user satisfaction, cost per task) |
| Performance tracking | Measure quality, latency and cost continuously, and evaluate outputs against a test set |
| Drift monitoring | Watch for data drift (inputs change) and model drift (performance degrades as the world changes) |
| Versioning | Track model and prompt versions so you can compare and roll back |
| Automatic model upgrades | Managed models can move to newer versions. Test before relying on them, and pin versions where stability matters |
| Security patches and updates | Keep platforms and dependencies patched |
| Feature Store (in Agent Platform) | Central, consistent store of the features (prepared data inputs) used by models, reused between training and serving to reduce mismatch |
| Feedback loops | Collect user ratings and human review to improve prompts, data and models |
Evaluation methods: compare against reference answers, use human raters, use another model as a judge (with care), and run A/B tests. Agent Platform includes evaluation services for this.
Prompt engineering
Prompt engineering is writing and refining the input to an LLM to get more accurate, relevant and useful output. It matters because the model has no other way to know what you want, it costs nothing in training, and small wording changes can change results a lot.
What makes a good prompt
- A clear task and the goal.
- Relevant context and background.
- A role or persona when useful.
- The desired format (table, bullets, JSON, length).
- Constraints (audience, tone, things to avoid).
- Examples of the output you want.
- Iteration: test, review, refine.
Techniques
| Technique | What you do | Use when |
|---|---|---|
| Zero-shot | Give the task with no examples | Simple, common tasks the model already handles |
| One-shot | Give one example | The format needs a single demonstration |
| Few-shot | Give several examples | You need consistent format, style or tricky classification |
| Role prompting | Tell the model to act as a specific role (“You are a senior tax adviser”) | You want a tone, expertise level or point of view |
| Prompt chaining | Break a job into steps, with each step’s output feeding the next prompt | Complex tasks: outline, then draft, then edit |
| Chain-of-thought (CoT) | Ask the model to reason step by step before answering | Maths, logic, multi-step reasoning |
| ReAct (Reason + Act) | Model alternates thinking and acting (calling a tool, searching), observing results, then continuing | Agents that must look things up or use tools while reasoning |
Notes:
- Few-shot beats zero-shot when format matters, because examples show the pattern.
- CoT improves reasoning accuracy and makes the logic visible. Some modern models reason on their own (“thinking” modes).
- ReAct is the basis of many agents: thought, action, observation, repeat.
- Prompt chaining is a workflow of several prompts. CoT is one prompt that reasons in steps. Don’t confuse them.
- Prompts are not permanent: models change, so re-test.
Grounding
Grounding connects a model’s answer to verifiable sources so it relies on real information rather than only memory. It reduces hallucinations, adds fresh data, and enables citations.
| Source of grounding | Meaning | Example |
|---|---|---|
| First-party enterprise data | Your own documents, databases and systems | HR policies, product catalogue, CRM |
| Third-party data | Data from outside providers you connect | Licensed market data, partner feeds |
| World data | Public, up-to-date world knowledge | Grounding with Google Search for news, prices, events |
Retrieval-augmented generation (RAG)
RAG is the main grounding technique.
- Your content is split into chunks and turned into embeddings (numeric meaning vectors) stored in a vector index.
- When a user asks a question, the system retrieves the most relevant chunks.
- The chunks are added to the prompt as context.
- The model generates an answer based on them, often with citations.
Effects on output: more accurate and specific, uses current and private data, can show sources, fewer hallucinations, and no retraining needed. Limits: quality depends on the data and retrieval. Poor or stale documents give poor answers, and the model can still misread what it retrieves.
Google Cloud grounding offerings
| Offering | Description |
|---|---|
| Pre-built RAG with Agent Search | Managed search and RAG over your data. Fastest route |
| RAG APIs | Building blocks (for example RAG Engine, ranking and grounding APIs) for custom RAG |
| Grounding with Google Search | Lets Gemini use live Google Search results and return cited answers for world knowledge and recent events |
Pick by data: your private data means Agent Search or RAG APIs; public, current world facts means Grounding with Google Search.
Sampling parameters and settings
These control how the model picks its next token. They shape randomness, length and safety, without changing the model.
| Setting | Effect | Guidance |
|---|---|---|
| Temperature | Controls randomness. Low (near 0) is focused and repeatable. High gives more varied, creative output | Low for facts, extraction, support answers. High for brainstorming and creative writing |
| Top-p (nucleus sampling) | Model chooses only from the smallest set of likely tokens whose combined probability reaches p (for example 0.9) | Lower p means safer, narrower output. Higher p means more variety |
| Top-k | Chooses only from the k most likely tokens | Lower k means more predictable |
| Max output tokens / output length | Caps how long the answer can be. Too low cuts the answer off mid-sentence | Set to the task. Also controls cost |
| Token count | Tokens in the input and output. Billing and context limits count in tokens | Shorter prompts cost less; long inputs must fit the context window |
| Stop sequences | Text that tells the model to stop | Control format |
| Safety settings | Thresholds that block content in harm categories (harassment, hate, sexually explicit, dangerous) | Tighten for public-facing apps |
Traps:
- High temperature is not a way to fix hallucinations; it raises them. Lower temperature and grounding help.
- Temperature 0 gives the most predictable output, not guaranteed correct output.
- Changing temperature does not add knowledge. Only grounding or training does.
- Sampling settings are not safety systems on their own. Use safety settings and filters for that.
Putting it together
| Scenario | Answer |
|---|---|
| Chatbot invents return policy details | Ground on the company’s policy documents with RAG, lower temperature |
| Assistant does not know this week’s news | Grounding with Google Search |
| Model must write in the brand voice consistently | Few-shot examples first; fine-tune if still short |
| Complex multi-step analysis comes out wrong | Chain-of-thought or prompt chaining |
| Agent must search then compute then answer | ReAct style with tools |
| Medical summaries before patient use | Human in the loop plus grounding |
| Quality worsens over months | Drift monitoring, evaluation and refreshed data |
| Ad copy needs variety | Raise temperature or top-p |
Exam checklist
- List the six limitations: data dependency, knowledge cutoff, bias, fairness, hallucination, edge cases.
- Give the fix for each: grounding, RAG, prompt engineering, fine-tuning, HITL.
- Explain monitoring terms: KPIs, drift, versioning, auto upgrades, patches, Feature Store.
- Define zero-shot, one-shot, few-shot, role prompting, prompt chaining, CoT and ReAct.
- Explain how RAG works and what it changes.
- Separate first-party, third-party and world data grounding.
- Name the three Google grounding offerings.
- Explain temperature, top-p, top-k, max output tokens and safety settings.