Exploring data transformation

Why data is valuable

• Real-time business insight: see what is happening now, not last month.
• Trends and patterns: forecast demand, spot churn, find fraud.
• Strategic decisions: base choices on evidence.
• Fuel for AI: models are only as good as the data they learn from. Organizations that own good data have an advantage.

Data is an asset only when it can be found, trusted and used. Collecting it without governance or analysis creates cost and risk, not value.

Databases, data warehouses and data lakes

Trap: BigQuery is a warehouse for analytics, not a database to run a shopping cart. Cloud SQL or Spanner serve transactions. Many modern designs blur the lines with a “lakehouse” (open formats in Cloud Storage queried by BigQuery), but for the exam keep the basic split.

Types of data

By source

By structure

Most enterprise data is unstructured. AI, especially generative AI, makes unstructured data far more useful.

The data supply chain

Think of data as moving through stages, like goods in a supply chain.

1. Data genesis: where data is created (customers, devices, apps, transactions).
2. Data collection: gathering it (ingestion from apps, sensors, streams, batch loads). Pub/Sub often sits here.
3. Data processing: cleaning, transforming and enriching (Dataflow, Spark).
4. Data storage: keeping it in the right place (Cloud Storage, BigQuery, databases).
5. Data analysis: exploring it with SQL, BI and ML (BigQuery, Looker, BigQuery ML).
6. Data activation: putting insight to work: dashboards, alerts, personalised experiences, automated decisions, feeding apps and AI.

The last step matters most to business leaders. Insight that no one acts on has no value.

Data governance

Data governance is the set of policies, roles and processes that keep data accurate, secure, private, usable and compliant across its life.

It covers:

• Ownership and stewardship: who is accountable for each data set.
• Data quality standards.
• Access control: who may see what (least privilege).
• Privacy and compliance: GDPR, HIPAA and similar rules; classification of sensitive data.
• Lineage and cataloging: knowing where data came from and what it means.
• Retention and lifecycle rules.

Why it matters: trust in data, regulatory compliance, reduced risk of breaches, and reliable AI. Google Cloud tools that support governance include Dataplex Universal Catalog (discover, catalog and govern data across stores), Sensitive Data Protection (find and mask sensitive data) and IAM.

Openness and interoperability

A data silo is data trapped in one system or department. Vendor lock-in is being unable to leave a provider without heavy cost.

Why openness matters:

• Open formats and standards (SQL, Parquet, Apache Iceberg, open source engines) let different tools read the same data.
• Silos disappear when data from many sources and clouds can be analysed together.
• Lock-in risk falls because data and code can move.

Google Cloud examples: BigQuery Omni analyses data sitting in other clouds, BigQuery reads open table formats, and managed open source engines (Spark, Kafka, PostgreSQL compatibility in AlloyDB) keep skills and code portable.

Choosing a Google Cloud data product

Key terms

Product matching

Quick decision path:

1. Analytics over huge data and reporting -> BigQuery.
2. Files, media, archives, raw lake data -> Cloud Storage.
3. Transactions, relational and familiar -> Cloud SQL (or AlloyDB for higher performance).
4. Relational but global scale and always on -> Spanner.
5. Huge key-based throughput, IoT or time series -> Bigtable.
6. App data with flexible documents and real-time sync -> Firestore.

Common trap: choosing Spanner when Cloud SQL is enough. Spanner is for global scale and very high availability; Cloud SQL is cheaper and simpler for ordinary apps.

Cloud Storage classes

All classes share the same durability, API and millisecond access. The differences are price to store, price to access, and minimum storage duration.

The colder the class, the cheaper to store but the more expensive to retrieve. “Retrieve rarely, keep for years, lowest storage cost” means Archive. “Access pattern unknown or changing” means Autoclass. Object Lifecycle Management rules can also move or delete objects on a schedule.

Trap: colder classes are not slower to read. Even Archive returns data in milliseconds; you pay with higher access fees and minimum duration.

Migrating and modernizing databases

Options to bring a database to the cloud:

Benefits of managed databases: automated backups, patching, replication and failover, so teams spend time on applications. Database Migration Service performs minimal-downtime migration by continuously replicating changes.

Smart analytics and business intelligence

Looker

Looker is Google Cloud’s enterprise BI and analytics platform. Key ideas:

• A semantic layer (LookML) defines metrics once, so everyone uses the same definition of “revenue”. This gives a single source of truth.
• It democratizes data: business users explore data and build dashboards through self-service, without asking an analyst or writing SQL.
• It queries data in place (including BigQuery), so reports reflect live data rather than stale extracts.
• It supports embedded analytics and pushing insight into workflows, such as alerts and actions in business apps.
• Conversational analytics with Gemini lets people ask questions in natural language.

Looker Studio is the lighter, free tool for simple dashboards and reports; Looker is the governed enterprise platform. For the exam, “governed, consistent metrics, embedded in apps” means Looker.

Real-time streaming analytics

Streaming analytics processes data as it arrives instead of in nightly batches.

Why it is critical:

• Detect fraud while the transaction is happening.
• Personalise the shopping experience during the visit.
• Monitor equipment and react before failure.
• Update dashboards and supply chains live.

Batch answers “what happened”. Streaming answers “what is happening”. Late insight often loses its value.

Modernizing data pipelines

A typical streaming pipeline: devices or apps -> Pub/Sub -> Dataflow -> BigQuery -> Looker dashboard.

Choosing: new pipeline, serverless, streaming or batch = Dataflow. Existing Spark or Hadoop code = Managed Service for Apache Spark. Decoupled event ingestion = Pub/Sub.