Optimizing performance and cost

Collecting performance information

Application performance monitoring

Method: define the SLI and a performance baseline; load test before release; compare before and after for each release (canary analysis); look at percentiles, not averages; relate utilisation to saturation (a service at 20% average CPU may still be throttled or lock-bound). Profile in production because it’s cheap and representative.

Active Assist and recommenders

Active Assist is Google Cloud’s suite of recommendation and insight engines. Each recommender analyses usage and posts recommendations in the Recommendation Hub (and via the Recommender API, exportable to BigQuery, notifications on change).

Facts: recommendations come with an estimated impact (savings or risk), can be applied, dismissed or marked claimed, and some can be applied through API or Terraform. VM rightsizing uses at least a week of Ops Agent or hypervisor utilisation (memory data improves quality when the agent is installed). Recommenders for Policy Troubleshooter, Policy Simulator and Network Analyzer are Active Assist insights too. Review them regularly, assign owners, and automate the safe ones (for example snapshot and delete idle disks). Gemini Cloud Assist can summarise them.

FinOps practices

FinOps is the practice of bringing financial accountability to cloud spend, in three loops: inform (visibility and allocation), optimise, and operate (governance, forecasting, culture). Engineers and finance share the work.

Visibility and allocation

• Labels (env, team, app, cost-center) on resources, enforced with organization policy or IaC; projects and folders per team or environment; tags. Labels appear in billing data; unlabelled spend is unowned spend.
• Cloud Billing reports and cost table, billing export to BigQuery (standard usage cost, detailed usage cost with resource-level data, pricing data) for custom analysis, joined with labels. Use Looker Studio dashboards. FinOps hub in the console gathers optimisation opportunities, utilisation of commitments and waste.
• Budgets and alerts on a billing account or scope (project, service, label): thresholds on actual or forecasted spend (50/90/100%); alerts by email, or by Pub/Sub to automate action (notify a team, throttle, in extreme cases disable billing on a sandbox project through a function; disabling billing stops services, so avoid it on production). Budgets don’t cap spend by themselves.
• Quotas and per-API limits cap runaway usage (a developer launching 1,000 VMs); custom quota overrides per project are a cost safety net.
• Anomaly detection surfaces unusual cost spikes.

Committed use and automatic discounts

Choose commitments for the steady baseline that will run for the term; use autoscaling and Spot for variable load; don’t commit to more than the stable minimum. Review commitment utilisation and coverage in the FinOps hub.

Spot VMs

• Spot VMs (--provisioning-model=SPOT) are surplus capacity at 60 to 91% off. Compute Engine can preempt (stop or delete) them at any time with a 30-second warning (shutdown script), and there’s no maximum runtime (unlike the older preemptible VMs, which last at most 24 hours). No SLA, no live migration, and they aren’t covered by CUDs the same way.
• Set the termination action (STOP or DELETE). Use Spot for fault-tolerant, stateless, batch, CI, rendering, data processing, and mix with on-demand for the critical base: MIG with Spot and on-demand instances, GKE Spot node pools (taint them; use Pod anti-affinity, PDBs and graceful termination), GKE Autopilot Spot Pods, Dataflow flexible resource scheduling, Cloud Batch.
• Design for interruption: checkpoint work, idempotent jobs, retry, spread across zones and machine types, handle the shutdown signal.
• Cloud Build and CI runners can use Spot-backed pools to cut cost.

Network cost and tiers

Optimizing observability costs

Observability can be a top-three line item. Reduce it without losing the signals that protect the SLO:

• Logging: exclusion and sampling filters, drop debug logs in production, reduce Data Access and VPC Flow Log volume (sample 5 to 10%, longer aggregation interval, only the subnets that matter), route archives to Cloud Storage, shorten retention, avoid logging full payloads in hot paths. Free tier: first 50 GiB per project per month is free to ingest.
• Metrics: avoid high-cardinality labels (user ID, request ID, full URL), drop unused metrics (in Prometheus, use relabelling or metric exclusion), increase the scrape interval (for example 60 seconds rather than 15), aggregate before sending (OTel Collector), use Prometheus recording rules carefully. Managed Prometheus is billed per sample, custom metrics per volume; system metrics from Google Cloud are mostly free.
• Traces: sampling rate, tail-based sampling for errors and slow requests, shorter span attributes. Cloud Trace charges per span ingested after a free allowance.
• Alerting: fewer, aggregated conditions (charged per condition and series evaluated), remove stale policies.
• Governance: budgets and alerts specific to the observability services, dashboards of bytes ingested per source (logging.googleapis.com/billing/bytes_ingested and metrics volume), owners per noisy source, and a review cadence.
• Don’t “save” by turning off Admin Activity or audit logs (not possible) or by sampling out the errors you need.

Optimizing individual workloads

Compute Engine

• Right-size from recommender and monitoring data; use custom machine types or the right family; consider E2 (cost-optimised) for general purpose; Arm (Axion/Tau T2A) for efficiency where software supports it; shut down idle VMs on schedules (instance schedules with resource policies start and stop dev environments overnight); delete idle disks, snapshots and IPs; pick balanced or standard persistent disks over SSD where IOPS don’t need it; use Hyperdisk performance tuning separate from capacity.
• Use MIG autoscaling so you pay for what the load needs; Spot for fault-tolerant tiers; CUDs for the baseline; images and snapshot lifecycle policies.

Google Kubernetes Engine

• Autopilot bills per Pod resource request (no idle node cost, managed bin packing); Standard bills nodes, so bin packing and utilisation are your job.
• Right-size requests and limits (use VPA recommendations, GKE workload rightsizing insights and the cost optimisation dashboard). Over-requested Pods strand node capacity; under-requested Pods get throttled or OOM-killed.
• Cluster autoscaler with the optimize-utilization profile and node auto-provisioning to pick the right machine shape; HPA for replicas; scale dev clusters down or to zero at night; Spot node pools for fault-tolerant workloads; CUDs for the baseline; multi-tenant clusters to share control plane cost; cost allocation (enable GKE cost allocation to attribute cost by namespace and label).
• Choose regional vs zonal deliberately (regional clusters cost more in cross-zone traffic and nodes).

Cloud Run

• Billed on CPU, memory and requests (request-based) or while the instance is up (instance-based, CPU always allocated). Choose request-based for spiky or intermittent traffic and instance-based for steady or background processing.
• Concurrency: higher concurrency means fewer instances (cheaper) if the code is thread-safe and CPU permits; min instances reduce cold starts but cost idle time (use startup CPU boost and lighter images first); max instances cap cost and protect downstream; right-size CPU and memory; keep images small; use committed-use (spend-based) discounts for steady usage; scale to zero in dev.

Other cost levers

• Storage: lifecycle rules and Autoclass in Cloud Storage; archive old logs and backups; delete unused snapshots.
• BigQuery: partition and cluster tables, avoid SELECT *, use slot reservations (editions) for steady load, set maximum bytes billed limits.
• Databases: right-size Cloud SQL, use read replicas only when needed, stop dev instances, use committed use on Cloud SQL.
• Idle projects: use unattended project recommender to find and shut them down; remove sandbox sprawl with TTL labels.

Cost and performance for AI workloads (Provisioned Throughput and tokens)

For generative AI on the Gemini Enterprise Agent Platform (formerly Vertex AI):

Optimising token consumption:

• Shorter, tighter prompts and system instructions; remove redundant context; summarise long conversation history; limit retrieved RAG chunks to what’s needed.
• Context caching for large repeated prefixes (documents, long instructions) at a reduced rate for cached tokens.
• Use the smallest model that meets quality (Flash and Flash-Lite class models for classification and routing, a larger model only where needed); model routing.
• Cap max output tokens and ask for structured, concise responses; avoid unnecessary reasoning or thinking budget where not needed.
• Avoid agent loops with unbounded tool calls; set step limits; cache tool results.
• Monitor tokens as a metric (input, output, per request, per agent and user), set budgets and quota alerts, and alert on anomalies.
• Reliability link: Provisioned Throughput for agent reliability, retries with backoff, and fallbacks to another model or region.