Arki1 - Google Cloud Authorized Training Partner
Data & Analytics

Introduction to Data Engineering on Google Cloud

In this course, you learn about data engineering on Google Cloud, the roles and responsibilities of data engineers, and how those map to offerings provided by Google Cloud. You also learn about ways to address data engineering challenges.

Duration 8h
Level Introductory
Modality In-Company / Live Online
Instructor Google Cloud Certified

Course Overview

In this course, you learn about data engineering on Google Cloud, the roles and responsibilities of data engineers, and how those map to offerings provided by Google Cloud. You also learn about ways to address data engineering challenges.

What You Will Learn

  • Understand the role of a data engineer.
  • Identify data engineering tasks and core components used on Google Cloud.
  • Understand how to create and deploy data pipelines of varying patterns on Google Cloud.
  • Identify and utilize various automation techniques on Google Cloud.

Target Audience

  • Data engineers
  • Database administrators
  • System administrators

Detailed Syllabus

Module 1: Data Engineering Tasks and Components Module 1
  • The Role of a Data Engineer
  • Data Sources Versus Data Sinks
  • Data Formats
  • Storage Solution Options on Google Cloud
  • Metadata Management Options on Google Cloud
  • Sharing Datasets Using Analytics Hub
  • Lab, Loading Data into BigQuery
Module 2: Data Replication and Migration Module 2
  • Replication and Migration Architecture
  • The gcloud Command Line Tool
  • Moving Datasets
  • Datastream
  • Lab, Datastream: PostgreSQL Replication to BigQuery
Module 3: The Extract and Load Data Pipeline Pattern Module 3
  • Extract and Load Architecture
  • The bq Command Line Tool
  • BigQuery Data Transfer Service
  • BigLake
  • Lab, BigLake: Qwik Start
Module 4: The Extract, Load, and Transform Data Pipeline Pattern Module 4
  • Extract, Load, and Transform (ELT) Architecture
  • SQL Scripting and Scheduling with BigQuery
  • Dataform
  • Lab, Create and Execute a SQL Workflow in Dataform
Module 5: The Extract, Transform, and Load Data Pipeline Pattern Module 5
  • Extract, Transform, and Load (ETL) Architecture
  • Google Cloud GUI Tools for ETL Data Pipelines
  • Batch Data Processing Using Dataproc
  • Streaming Data Processing Options
  • Bigtable and Data Pipelines
  • Lab, Use Dataproc Serverless for Spark to Load BigQuery
  • Lab, Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow
Module 6: Automation Techniques Module 6
  • Automation Patterns and Options for Pipelines
  • Cloud Scheduler and Workflows
  • Cloud Composer
  • Cloud Run Functions
  • Eventarc
  • Lab, Use Cloud Run Functions to Load BigQuery
Corporate Solutions

Request In-Company Training

Private cohort scheduled for your engineering team with customized hands-on labs and enterprise pricing.

  • Flexible dates and times for your team
  • Hands-on labs in official Google Cloud environment
  • Official certificates of completion issued
Request In-Company Proposal
Enterprise Solutions

Accelerate Your Cloud Transformation

Talk directly with our curriculum advisors to design a custom training track for your team.