Integrating Google Cloud services

Data and storage connections

Cloud SQL and AlloyDB

• Connect with the connectors: language Cloud SQL Connectors (Java, Python, Go, Node.js), the Cloud SQL Auth Proxy, or Cloud Run’s built-in Cloud SQL connection. All use IAM to authorise, encrypt traffic, and avoid IP allow lists and SSL certificate management. AlloyDB has the AlloyDB Auth Proxy and connectors in the same style.
• Required role for the caller: Cloud SQL Client (roles/cloudsql.client). For passwordless access enable IAM database authentication (needs Cloud SQL Instance User role).
• Paths to a database: public IP (with authorised networks, avoid), private IP (VPC peering or Private Service Connect; reach from Cloud Run through Direct VPC egress or a connector), or the proxy.
• On Cloud Run, --add-cloudsql-instances exposes a Unix socket at /cloudsql/PROJECT:REGION:INSTANCE; connect with that path as the host.
• Connection management: databases have a connection limit. Use a connection pool per instance, create it once at startup (global scope, not per request), keep pool size small, and tune max instances x concurrency x pool size below the database limit. Handle dropped connections with retries and reconnect, and set connection and idle timeouts. For many short-lived clients consider a managed pooler (Managed Connection Pooling or PgBouncer).
• Lazy initialisation: in Cloud Run functions, initialise clients at global scope and reuse them across invocations; do heavy init lazily to protect cold starts.
• Run schema migrations as a Cloud Run job or pipeline step, not on every instance start.

Firestore

• Native mode uses client libraries (server SDKs with IAM, mobile/web SDKs with security rules and Firebase Authentication). Use the Admin/server library from Cloud Run and GKE.
• Operations: set, add, get, update, delete; queries with where, orderBy, limit; cursors for pagination (startAfter) rather than offsets, which still read skipped documents.
• Transactions (read then write, atomic, retried on contention) and batched writes (writes only, atomic). BulkWriter is for high-volume writes with rate ramp-up.
• Real-time listeners (onSnapshot) push changes to clients; offline persistence in mobile and web SDKs.
• Composite indexes are needed for multi-field queries; a failed query error returns a link to create the index. Write rate on a single document is limited (about one sustained write per second), so shard counters across documents (distributed counter).
• Use TTL policies to expire documents automatically, and keep reads cheap (billing is per document read).

Spanner and Bigtable

• Spanner client libraries manage a session pool; create the client once. Use read-write transactions (the library retries on aborts, so the function body must be idempotent and side-effect free), read-only transactions and stale reads for lower latency; mutations for bulk writes versus DML for SQL; partitioned DML for large updates and deletes.
• Bigtable: create the client once; read by row key or key range; use batching (MutateRows) and filters; put effort in the row key design; app profiles choose single-cluster or multi-cluster routing.

Cloud Storage

• Client libraries: upload and download objects, set metadata, list with prefixes, delete. Use resumable uploads for large files, parallel composite uploads and the gcloud storage tools for speed.
• Preconditions: ifGenerationMatch (0 means “only if it does not exist”) prevents overwriting and races; generations identify versions.
• Notifications: object events (finalize, delete, archive, metadata update) go to Pub/Sub, or through Eventarc to Cloud Run and functions.
• Signed URLs let clients upload and download directly; keep expiry short (V4 maximum is 7 days).
• Choose bucket location and storage class by access pattern: Standard, Nearline (30 days), Coldline (90 days), Archive (365 days) minimum storage durations; Autoclass moves objects automatically.

Messaging with Pub/Sub

Concepts

• A topic receives messages from publishers; subscriptions deliver to consumers. Each subscription gets every message; consumers sharing a subscription split them (competing consumers).
• Delivery: at-least-once by default. Duplicates and out-of-order delivery can occur, so handlers must be idempotent.
• Subscription types: pull (client asks; use streaming pull for throughput), push (Pub/Sub POSTs to an HTTPS endpoint, with an OIDC token for authentication), BigQuery subscription (writes rows directly), Cloud Storage subscription (writes files).
• Acknowledgement deadline: 10 seconds by default (range 10 to 600). If you do not ack in time, the message is redelivered; extend the deadline with modifyAckDeadline (client libraries do this with lease management).
• Retention: unacknowledged messages are kept 7 days by default (configurable up to 31 days). Topic retention can store messages longer for replay; seek to a timestamp or snapshot to replay.
• Dead-letter topic: after a set number of delivery attempts (5 to 100) a message goes to a dead-letter topic for inspection. Retry policy: exponential backoff between redeliveries (minimum and maximum backoff) instead of immediate retry.
• Ordering: set an ordering key on messages and enable message ordering on the subscription; messages with the same key in the same region are delivered in order. Ordering lowers parallelism per key.
• Exactly-once delivery: available on pull subscriptions (regional); acks carry a success confirmation and acknowledged messages are not redelivered. Still design idempotent handlers for end-to-end safety (the business effect may happen elsewhere too).
• Filtering: a subscription filter on message attributes drops unwanted messages without cost to the consumer.
• Schemas: attach an Avro or Protobuf schema to a topic to validate messages.
• Message limits: up to 10 MB per message; attributes carry metadata. Publishers batch messages for throughput (batch size, count and delay settings).

Publishing and consuming code

• Create the publisher or subscriber client once. Publish returns a future; handle failures (the library retries transient errors). Set batching for throughput or low latency, and ordering keys with resume-on-error logic if ordered.
• Consume with flow control (limit outstanding messages and bytes) so a slow handler is not overwhelmed. Ack only after successful processing; nack on retryable failures. Poison messages go to the dead-letter topic.
• For serverless consumers use push to Cloud Run or Eventarc; for long-running workers use pull and scale on backlog (num_undelivered_messages, oldest unacked message age).

Consuming Google Cloud APIs

Enabling services and ways to call APIs

• Enable the API first: gcloud services enable SERVICE.googleapis.com (or the console). Calls to a disabled API fail with a 403 that says so. Some APIs need billing enabled and have their own quotas.
• Cloud Client Libraries are the recommended choice: idiomatic, use ADC, include retries, pagination and auth. Use the lower-level REST (JSON over HTTP) or gRPC (Protocol Buffers, streaming, lower overhead) APIs when no library exists or you need specific control. The API Explorer is a browser tool to try REST methods quickly; APIs Explorer shows request and response shape. Older Google API Client Libraries exist for REST discovery docs; prefer Cloud Client Libraries where available.
• Authentication: libraries find credentials with ADC; for raw REST you send an Authorization: Bearer access token (gcloud auth print-access-token). API keys identify a project for some public data APIs and are not for authorising users or servers to private resources.

Efficient and resilient calls

Backoff rules: retry only transient errors (HTTP 408, 429, 500, 502, 503, 504; gRPC UNAVAILABLE, DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED when appropriate). Do not retry permanent client errors such as 400, 401, 403 or 404 without fixing the cause. Wait base x 2^attempt plus random jitter so clients do not retry in lockstep. Only retry idempotent operations (or use idempotency tokens). 429 means quota or rate exceeded: slow down, request a quota increase if sustained.

Service accounts for API calls

• A service account is the identity of a workload. Attach it to Cloud Run, GKE (through Workload Identity Federation for GKE), Compute Engine, Cloud Build or functions, and the libraries obtain short-lived tokens from the metadata server through ADC.
• Default service accounts exist (Compute Engine default, App Engine default) and may have broad roles; create user-managed service accounts with narrow roles per workload.
• Impersonation: a principal with roles/iam.serviceAccountTokenCreator on a service account can mint tokens as it; roles/iam.serviceAccountUser allows attaching it to a resource. Use for testing with production-like permissions and for cross-project access.
• Avoid downloaded keys; if unavoidable, rotate and restrict them. Outside Google Cloud use Workload Identity Federation.
• Scopes versus IAM: for most services IAM roles decide access; legacy access scopes on Compute Engine VMs can still limit it (use cloud-platform scope with IAM).

Troubleshooting and observability

The operations suite is Google Cloud Observability (formerly Cloud Operations or Stackdriver).

Instrumenting code

• Logs: write to stdout/stderr. Cloud Run, GKE and functions forward them automatically. Emit structured JSON (one object per line) with severity, message and custom fields so they are indexed (jsonPayload). Special fields: logging.googleapis.com/trace (value projects/PROJECT_ID/traces/TRACE_ID), logging.googleapis.com/spanId, labels and httpRequest. Setting the trace field correlates logs with the trace and groups a request’s logs together.
• Metrics: use system metrics (request count, latency, CPU) and define custom metrics or log-based metrics for business signals. Alert on symptoms users feel (error rate, latency) and on saturation.
• Traces: use OpenTelemetry instrumentation. Cloud Trace reads context headers: X-Cloud-Trace-Context: TRACE_ID/SPAN_ID;o=1 and the W3C traceparent. To follow a request across services, propagate the incoming trace header on every outbound call; services then create spans with the same trace ID, so you can view the whole call chain and see which hop is slow. Cloud Run and load balancers add trace context for incoming requests automatically.
• Error Reporting: errors appear automatically when logs contain stack traces (exceptions logged with severity ERROR) or via the client library; include serviceContext (service and version) to group by release. Create notifications for new error groups.

Finding and fixing issues

1. Alert fires (Monitoring) on error rate or latency SLO burn.
2. Open the service dashboard; find the time window and affected revision.
3. In Logs Explorer filter by resource.type, severity>=ERROR, the trace ID; follow the trace across services.
4. Use Trace to find the slow span (downstream call, DB query) and Profiler to find hot functions.
5. Check Error Reporting for the exception and first-seen release; roll back through traffic splitting if a new revision caused it.
6. Fix, add a test and alert so it does not recur.

Common causes: exhausted connection pools, missing IAM role (403), disabled API, cold starts, quota (429), memory limit (OOM) and unhandled retries amplifying load.

AI-assisted observability

Gemini Cloud Assist and Gemini in Logs Explorer and Monitoring let you ask questions in natural language, generate log queries and PromQL, summarise or explain log entries, and run investigations that correlate logs, metrics, traces and recent changes to propose root causes and fixes. They speed triage; verify suggestions against real telemetry before acting, and remember data access permissions still apply.