Collaborating to manage data and models

Exploring and organising data

Where the data lives

Organising data for efficient training

• Prefer columnar and binary formats at scale: Parquet or Avro for tabular; TFRecord for TensorFlow input pipelines. CSV and JSON are slow to parse.
• Shard large files (hundreds of MB each) rather than millions of tiny files. Thousands of small objects starve accelerators, and Cloud Storage handles throughput better with parallel reads over large objects.
• Co-locate the bucket and the training job in the same region to avoid latency and egress fees. Use a regional or dual-region bucket, not multi-region, for training data used in one place.
• Images/video/audio: keep binaries in Cloud Storage and a manifest (CSV or JSONL) of URIs and labels. Vertex AI datasets use import files this way.
• Text: JSONL records, one document per line, with the label or prompt/response fields.
• Tabular: partition and cluster BigQuery tables by time or key so training reads only the slice it needs. Use TABLESAMPLE or hashed splits (MOD(FARM_FINGERPRINT(id), 10)) for repeatable train/validation/test splits. Hashing on the entity (user ID) keeps one entity in one split.
• Keep raw, cleaned and feature layers separate and versioned.

Vertex AI managed datasets

A managed dataset wraps data (image, text, video or tabular) with a schema, labels and splits, and plugs into AutoML and custom training.

• Create from Cloud Storage import files or BigQuery tables. Datasets are regional.
• Dataset versioning and dataset annotation sets let you track label changes. Data labelling jobs send items to human labellers with instructions.
• Stats and visualisations help spot missing values, skew and imbalance before training.
• Training/validation/test split: set fractions or a predefined split column. Never evaluate on training rows.
• Data Catalog / Dataplex Universal Catalog records lineage and metadata for discovery across projects, and BigQuery sharing (formerly Analytics Hub) shares datasets without copying them.

Data preprocessing

Preprocessing choices that the exam probes:

• Missing values: impute (mean/median/mode, model-based) or add an “is-missing” indicator. Don’t silently drop many rows.
• Scaling: standardise or normalise numeric features for gradient-descent and distance models. Trees don’t need it.
• Categorical: one-hot for few categories, embeddings or hashing for high cardinality.
• Skewed numerics: log transform or bucketise.
• Imbalanced classes: resample, class weights, or an appropriate metric (precision-recall AUC, F1) rather than accuracy.
• Leakage: features that wouldn’t exist at prediction time, or statistics computed over the full dataset before the split. Fit transforms on the training split only.
• Training-serving skew appears when preprocessing is rewritten separately for serving. Fix by sharing one transformation (TFT, a TRANSFORM clause, a shared container, or a feature store).

Vertex AI Feature Store

A feature store is a central, governed place to define, compute, share and serve features.

• The current Vertex AI Feature Store is built on BigQuery: your feature data stays in BigQuery tables or views, and Feature Store adds an online-serving layer. (The older “legacy” Feature Store with its own Bigtable storage is deprecated. Be aware old materials describe it.)
• Concepts: a feature group (a BigQuery source with entity ID and optional feature timestamp column), features (columns), a feature view (a selection of features synced for online serving), and an online store instance that holds the synced values.
• Offline use: read features from BigQuery for training, with point-in-time correct joins so a training row only sees feature values known at that row’s timestamp. This prevents leakage.
• Online use: low-latency reads by entity ID at prediction time. Sync is scheduled or on demand from BigQuery to the online store. Choose Bigtable-backed or optimized online serving depending on latency and scale.
• Benefits: reuse across teams, consistency between training and serving (same definition), monitoring and lineage, and less duplicated pipeline code.
• Choose it when several models share features, you need low-latency lookups of precomputed aggregates, or skew has hurt you. For one model with features computed in the request, a feature store is overkill.

Privacy and sensitive data

Principles: collect the minimum needed, de-identify before it reaches the notebook or training set, restrict access by IAM and policy tags, document purpose and retention, respect data residency (choose regions deliberately), and remember that models can memorise training data, so scrub sensitive records before training rather than after. Unstructured data (documents, call transcripts) must be scanned too.

Ingesting text documents for inference

Generative and search workloads need documents turned into something a model can use:

• Document AI or Layout Parser extracts text and structure from PDFs and scans.
• Chunk the text, embed it (Vertex AI embedding models), and store vectors in Vector Search, BigQuery, AlloyDB or a Vertex AI Search data store.
• For batch inference over documents, put them in Cloud Storage or BigQuery and run a batch prediction or ML.GENERATE_TEXT job.
• For online inference over uploaded documents, send them to Gemini directly (multimodal input via Cloud Storage URI or inline) or process through a RAG endpoint.
• Streaming ingestion: Pub/Sub into Dataflow, which calls the embedding model and writes to the index.

Model prototyping in notebooks

Choosing the Jupyter backend

Workbench can attach to Dataproc clusters or Dataproc Serverless interactive sessions, so a notebook runs PySpark code on the cluster. Pick Spark kernels when the data is too big for a single VM, or when the team already has Spark code.

Security best practices for Workbench

• Run instances with a dedicated service account holding least-privilege roles, rather than the default Compute service account. Control user access with IAM (roles/notebooks.runner, iam.serviceAccountUser).
• No external IP, use Private Google Access or Private Service Connect, deploy in a VPC with firewall rules, and place the project in a VPC Service Controls perimeter.
• Use CMEK for disk encryption, Shielded VM options (secure boot, vTPM), and disable root access and file download/terminal where policy requires.
• Use Secret Manager for credentials, never keys in notebooks. Enable idle shutdown to save cost and reduce exposure. Use OS Login and organisation policies for constraints.
• Keep PII out of notebooks, using de-identified extracts and policy tags.

Source control and frameworks

• Workbench integrates with Git (the JupyterLab Git extension, GitHub and Cloud Source Repositories or other providers). Keep notebooks and shared modules in a repo, review them with pull requests, and move reusable code into packages and pipeline components. Strip outputs and avoid committing data.
• Prebuilt Deep Learning VM/Container images ship TensorFlow, PyTorch, scikit-learn, XGBoost, JAX and Spark with CUDA drivers. Choose the framework to match the team’s code and the accelerator (JAX and TensorFlow for TPUs; PyTorch with XLA on TPU too).
• Prototype on a sample, then move to scalable training: package code and run a Vertex AI custom training job (see the next domain).

Model Garden for prototyping

Browse, test in the console, and call Gemini, Imagen, partner and open models. Open models can be deployed to your own endpoint or opened in a notebook (sample notebooks). Use Agent Studio (previously Vertex AI Studio) to design and test prompts, compare models and export code. Use open-source models when you need local weights, custom licensing terms or tuning.

Tracking and running experiments

Choose by framework and need: TensorFlow/PyTorch training-curve debugging goes to TensorBoard. Comparing hyperparameters and metrics across runs goes to Experiments. Repeatable multi-step workflows go to Pipelines.

Reproducibility checklist: fix random seeds, record data version (BigQuery snapshot or table decorator, dataset version, Cloud Storage object generation), code version (commit hash), container image digest, hyperparameters and the environment. If any is missing you can’t rerun the experiment.

Evaluating generative AI solutions

Generative outputs have no single ground truth, so evaluation mixes methods:

• Gen AI evaluation service (Vertex AI): computation-based metrics (ROUGE, BLEU, exact match) where reference answers exist; model-based (LLM-as-judge) metrics such as fluency, coherence, groundedness, safety, instruction following; pairwise comparison between two models or prompts (autorater); custom rubrics; and for RAG, retrieval metrics (context precision/recall) and answer groundedness.
• Golden datasets of prompts with expected outputs are the regression suite. Run them whenever prompt, model or retrieval changes.
• Human evaluation for subjective quality and for calibrating autoraters.
• Track evaluation runs in Experiments.
• Responsible-AI checks: safety filters, toxicity, bias across groups, jailbreak/red-team prompts.
• Online, collect user feedback and sample traffic for review.