Architecting low-code AI solutions

The effort ladder

When a question says “the team has no ML expertise”, “minimal development effort” or “fastest time to market”, climb the ladder from the top and stop at the first rung that satisfies the requirement.

BigQuery ML

BigQuery ML (BQML) creates, evaluates and serves models with SQL inside BigQuery. Data never leaves the warehouse, which matters for governance and for avoiding export pipelines.

CREATE OR REPLACE MODEL `proj.ds.churn`
OPTIONS (model_type = 'LOGISTIC_REG', input_label_cols = ['churned']) AS
SELECT * FROM `proj.ds.customers`;

Choosing the model type

Generative use: create a remote model over a Gemini or other Vertex AI model, then call it with ML.GENERATE_TEXT or AI.GENERATE. Embeddings come from ML.GENERATE_EMBEDDING, followed by VECTOR_SEARCH for similarity. This is how you apply an LLM to rows in a table without moving data.

Feature engineering and selection in BQML

• Automatic preprocessing is on by default: numeric standardisation for some models, one-hot encoding of strings, imputation of nulls, timestamp feature extraction. Set ENABLE_GLOBAL_EXPLAIN and a TRANSFORM clause when you want to control the preprocessing yourself.
• The TRANSFORM clause records feature transformations inside the model, so ML.PREDICT applies the same preprocessing automatically. This is the BQML answer to training-serving skew.
• Manual functions (ML. preprocessing): ML.STANDARD_SCALER, ML.MIN_MAX_SCALER, ML.BUCKETIZE, ML.QUANTILE_BUCKETIZE, ML.POLYNOMIAL_EXPAND, ML.FEATURE_CROSS, ML.HASH_BUCKETIZE, ML.NGRAMS, ML.TF_IDF, ML.IMPUTER, ML.ONE_HOT_ENCODER, ML.LABEL_ENCODER.
• Selection and inspection: ML.FEATURE_INFO (min, max, mean, nulls per column), ML.WEIGHTS, ML.GLOBAL_EXPLAIN and ML.FEATURE_IMPORTANCE for tree models. Use L1 regularisation (L1_REG) to zero out weak features in linear models.
• Overfitting controls: L2_REG, EARLY_STOP, MIN_REL_PROGRESS, tree depth and subsampling.
• Class imbalance: AUTO_CLASS_WEIGHTS = TRUE.
• Hyperparameter tuning: NUM_TRIALS and a search space in the OPTIONS clause.
• Data split: DATA_SPLIT_METHOD of RANDOM, SEQ (ordered by a column, right for time-dependent data), CUSTOM (a boolean column), or NO_SPLIT. Splitting randomly on time series leaks the future.

Evaluating and predicting

• BQML predictions are batch by default. To serve online, register the model in Vertex AI Model Registry (model_registry = "VERTEX_AI" option) and deploy to an endpoint, or export the model to Cloud Storage as a TensorFlow SavedModel or similar.
• Models can be shared with IAM like tables (bigquery.dataViewer/dataEditor roles and dataset ACLs).
• Cost: models trained on-demand are billed by bytes processed; boosted trees, DNNs and AutoML-style types run on different pricing. Matrix factorisation needs reserved slots.

Trap: a question that asks for “real-time predictions below 100 ms” is not answered by ML.PREDICT alone. Export or register to Vertex AI and deploy an endpoint.

ML APIs and foundation models

Pre-trained APIs

Rules of thumb: if the output is generic (detect objects, transcribe a call, translate), use the API. If the labels are yours (“defect type A/B/C”), you need AutoML or custom training.

Model Garden

Model Garden is the catalogue of models in Vertex AI: Google’s Gemini, Imagen and embedding models, partner models (for example Anthropic Claude and Meta Llama as managed APIs), and open models (Gemma, Mistral, and others) you can deploy to your own endpoint or fine-tune.

• Model-as-a-service (MaaS): a managed API, pay per token, no infrastructure. Pick this for the least operational effort.
• Self-deploy: deploys an open model container to a Vertex AI endpoint on GPUs or TPUs. Pick this when you need control, private weights, specific licences or fine-tuning.
• Foundation model vs task-specific model: Gemini is multimodal and general. Smaller task-specific or open models cost less per call and fit latency-sensitive workloads.
• Choose by modality (text, image, audio, video), context window, latency and cost, languages, and data-residency or licence limits.

Controls worth knowing for generative apps: system instructions, temperature and top-p, safety filters, grounding (Google Search or your data), function calling, structured output (JSON schema), context caching to cut cost on repeated long prompts, and batch prediction for non-interactive volume.

Prompting, tuning and when to escalate

1. Prompt engineering and few-shot examples.
2. RAG or grounding for facts that change or are private.
3. Supervised fine-tuning (or parameter-efficient tuning such as LoRA) to change style, format or domain behaviour.
4. Preference tuning or distillation for refinement or to move quality into a smaller, cheaper model.

Fine-tune to change how a model behaves. Use RAG to change what it knows.

Industry-specific APIs

• Document AI extracts text, tables, key-value pairs and entities from documents. It offers general processors (OCR, form parser, layout parser), specialised processors (invoice, expense, ID, bank statement, contract, lending) and custom processors (custom extractor, classifier, splitter) trained on your documents or built on a generative model. Use human-in-the-loop review for low-confidence fields. Processors are versioned, and you can pin a version.
• Retail API (Vertex AI Search for commerce): product search and recommendations, with models such as “recommended for you”, “similar items” and “frequently bought together”, trained from user events and the product catalogue. Needs user events ingested continuously.
• Healthcare: Cloud Healthcare API (FHIR, HL7v2, DICOM) with de-identification, and healthcare-specific language models.
• Contact Center AI / Customer Engagement Suite: Dialogflow CX conversational agents, Agent Assist, Conversational Insights.
• Media and other: Video Intelligence and Transcoder, Vision Warehouse. Matches the question’s industry to the product.

Retrieval augmented generation with Agent Builder

RAG fetches relevant passages from your data at query time and supplies them to the model, so answers are grounded and up to date.

Pipeline: ingest documents (Cloud Storage, BigQuery, websites, Drive, third-party connectors) → parse and chunk → embed → index → at query time retrieve top-k → rerank (optional) → send passages plus question to the model → cite sources.

Decisions the exam likes: unstructured PDFs and HTML go to a unstructured data store; structured rows go to a structured store. Layout-aware parsing helps tables and scanned documents. Smaller chunks raise precision, larger chunks keep context. Hybrid search (keyword plus semantic) and reranking improve relevance. Apply document-level access control so users only retrieve what they may read. Hallucination when the knowledge base is correct points to retrieval or chunking, not a need to fine-tune.

AutoML

AutoML on Vertex AI trains models from your labelled data without you choosing architectures or tuning hyperparameters. You supply a managed dataset and a budget (node hours), and it handles search, training and evaluation.

Preparing data

• Labels: use Google’s data labelling (human labellers) or import labelled data. Label quality limits model quality. Consistent, well-defined classes matter more than volume.
• Minimums are small: roughly 100 examples per class for image classification, 1,000 rows for tabular, but more data and balanced classes give much better results. Use a spread of angles, lighting and backgrounds for images.
• Formats: tabular from BigQuery or CSV in Cloud Storage; images and video via an import file (JSONL/CSV) of URIs and labels; text as JSONL or CSV.
• Splits: automatic 80/10/10 by default, or set a split column manually. Use a time column for chronological splits.
• Feature selection: remove IDs, leakage columns (anything unknown at prediction time) and columns with mostly nulls. Set the right data type per column (categorical, numeric, text, timestamp). AutoML treats a numeric column as categorical only if you say so.

Data types and tasks

Tabular and Tabular Workflows

• AutoML Tabular picks among tree and neural models and ensembles them. Output includes feature importance (local and global) from Vertex Explainable AI.
• Tabular Workflows break AutoML into steps (feature engineering, architecture search, tuning, ensembling) that are Vertex AI Pipelines, so you can see, customise and rerun each stage. Variants include End-to-End AutoML, Wide & Deep, TabNet, Forecasting and Prophet. Pick them when you need pipeline visibility, a specific architecture, larger datasets (they support much bigger data than standard AutoML Tabular) or integration with MLOps tooling.

Forecasting

• Needs a time column, a target, and a time series identifier column when there are many series. Columns can be available at forecast time (promotions, holidays), unavailable at forecast time (past-only covariates), or attributes (static per series).
• Set the forecast horizon and context window (how much history to use). Granularity must match data (hourly, daily). Missing steps should be imputed or flagged.
• AutoML forecasting, Seq2Seq+ and TiDE models are available, plus BQML ARIMA_PLUS when data is in BigQuery and simple SQL suffices.
• Evaluate with metrics such as MAPE, RMSE and quantile loss, and always on a chronological hold-out.

Configuring and debugging trained models

• Budget: set the node-hour budget. Too low stops early with weaker models. Early stopping can end training sooner.
• Evaluation: confusion matrix, precision-recall and ROC curves, per-class precision and recall, and the confidence threshold slider. Raise the threshold for precision, lower it for recall.
• Poor accuracy: too little or imbalanced data, label noise, wrong data types, leakage (suspiciously perfect scores), distribution mismatch between training and production. Fix data first; adding budget rarely helps.
• Per-class weakness: add more examples of that class and look at the misclassified examples in the evaluation view.
• Explanations: enable feature attributions for tabular, and integrated gradients or XRAI for images.
• Deploy: to a Vertex AI endpoint for online use, batch prediction for volume, or export an edge model (image and video models can export to TF Lite or Core ML for devices).

Pick the right tool: common comparisons