Data & Analytics
Introduction to Data Engineering on Google Cloud
In this course, you learn about data engineering on Google Cloud, the roles and responsibilities of data engineers, and how those map to offerings provided by Google Cloud. You also learn about ways to address data engineering challenges.
Duration 8h
Level Introductory
Modality In-Company / Live Online
Instructor Google Cloud Certified
Course Overview
In this course, you learn about data engineering on Google Cloud, the roles and responsibilities of data engineers, and how those map to offerings provided by Google Cloud. You also learn about ways to address data engineering challenges.
What You Will Learn
- Understand the role of a data engineer.
- Identify data engineering tasks and core components used on Google Cloud.
- Understand how to create and deploy data pipelines of varying patterns on Google Cloud.
- Identify and utilize various automation techniques on Google Cloud.
Target Audience
- Data engineers
- Database administrators
- System administrators
Detailed Syllabus
Module 1: Data Engineering Tasks and Components Module 1
- The Role of a Data Engineer
- Data Sources Versus Data Sinks
- Data Formats
- Storage Solution Options on Google Cloud
- Metadata Management Options on Google Cloud
- Sharing Datasets Using Analytics Hub
- Lab, Loading Data into BigQuery
Module 2: Data Replication and Migration Module 2
- Replication and Migration Architecture
- The gcloud Command Line Tool
- Moving Datasets
- Datastream
- Lab, Datastream: PostgreSQL Replication to BigQuery
Module 3: The Extract and Load Data Pipeline Pattern Module 3
- Extract and Load Architecture
- The bq Command Line Tool
- BigQuery Data Transfer Service
- BigLake
- Lab, BigLake: Qwik Start
Module 4: The Extract, Load, and Transform Data Pipeline Pattern Module 4
- Extract, Load, and Transform (ELT) Architecture
- SQL Scripting and Scheduling with BigQuery
- Dataform
- Lab, Create and Execute a SQL Workflow in Dataform
Module 5: The Extract, Transform, and Load Data Pipeline Pattern Module 5
- Extract, Transform, and Load (ETL) Architecture
- Google Cloud GUI Tools for ETL Data Pipelines
- Batch Data Processing Using Dataproc
- Streaming Data Processing Options
- Bigtable and Data Pipelines
- Lab, Use Dataproc Serverless for Spark to Load BigQuery
- Lab, Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow
Module 6: Automation Techniques Module 6
- Automation Patterns and Options for Pipelines
- Cloud Scheduler and Workflows
- Cloud Composer
- Cloud Run Functions
- Eventarc
- Lab, Use Cloud Run Functions to Load BigQuery
Corporate Solutions
Request In-Company Training
Private cohort scheduled for your engineering team with customized hands-on labs and enterprise pricing.
- Flexible dates and times for your team
- Hands-on labs in official Google Cloud environment
- Official certificates of completion issued
Enterprise Solutions
Accelerate Your Cloud Transformation
Talk directly with our curriculum advisors to design a custom training track for your team.