About this course: This 1-week, accelerated on-demand course builds upon Google Cloud Platform Big Data and Machine Learning Fundamentals. Through a combination of instructor-led presentations, demonstrations, and hands-on labs, students learn how to carry out no-ops data warehousing, analysis and pipeline processing. Prerequisites: • Google Cloud Platform Big Data and Machine Learning Fundamentals • Experience using a SQL-like query language to analyze data • Knowledge of either Python or Java Notes: • You'll need a Google/Gmail account and a credit card or bank account to sign up for the Google Cloud Platform free trial (Google is currently blocked in China). • There is a known issue with certain EU countries where individuals are not able to sign up, but you may sign up as "business" status and intend to see a potential economic benefit from the trial. More details at: https://support.google.com/cloud/answer/6090602 • More Google Cloud Platform free trial FAQs are available at: https://cloud.google.com/free-trial/

Who is this class for: This class is intended for data analysts and data scientists responsible for: analyzing and visualizing big data, implementing cloud-based big data solutions, deploying or migrating big data applications to the public cloud, implementing and maintaining large-scale data storage environments, and transforming/processing big data.

Created by: Google Cloud

Taught by: Google Cloud Training

Basic Info	Course 3 of 5 in the Data Engineering on Google Cloud Platform Specialization.
Level	Intermediate
Commitment	1 week of study, 6-8 hours/week
Language	English
How To Pass	Pass all graded assignments to complete the course.
User Ratings	4.4 stars Average User Rating 4.4See what learners said

Preview

Syllabus

WEEK 1

Welcome to Serverless Data Analysis with Google BigQuery and Cloud Dataflow

2 videos

Video: Course Introduction
Video: Who is a Data Engineer?

Module 1: Serverless Data Analysis with BigQuery

59 videos, 3 readings

Video: Serverless Data Analysis with BigQuery Introduction
Video: Module 1 Agenda
Video: 2nd Generation of Big Data at Google
Video: Query Large Datasets
Video: Query Large Datasets (Example)
Video: BigQuery Offers...
Video: BigQuery is a great choice because...
Video: Example architecture for data analytics
Video: Project contains users, datasets
Video: Access control is via the dataset
Video: Tables and jobs
Video: BigQuery storage is columnar
Video: Queries and Functions Intro
Video: Running a Query from Web Console
Video: Query Syntax is SQL 2011 + Extensions
Video: Aggregate Functions and GROUP BY
Video: Subqueries Can Do Everything a Query Can
Video: Can Query from Multiple Tables
Video: JOIN ON Fields Across Tables
Video: Demo of JOIN ON Fields Across Tables
Video: Lab - Serverless Data Analysis (Java/Python) : Part 1
Reading: Codelab - Building a BigQuery Query
Video: Role of BigQuery in Data Processing
Video: Load Data Using CLI, Web UI, or API
Video: Loading Data Into BigQuery with Web UI
Video: Loading Data with bq Command-Line Tool
Video: Loading Data with Cloud Shell
Reading: Codelab - Loading and Exporting Data
Video: BigQuery Data Types - Standard SQL
Video: Joining Tables in BigQuery and WITH, COUNT(DISTINCT)
Video: COUNT (DISTINCT) Example Result
Video: ARRAY / STRUCT Example
Video: Aggregate Data in Arrays
Video: ARRAY / STRUCT Example
Video: Processing an Array
Video: JOIN Condition Example
Video: Standard SQL Functions
Video: Analytical Window Functions
Video: Analytical SQL and Window Functions
Video: Window Function Example
Video: Date and Time Functions
Video: User-defined Functions
Video: External User-defined Function
Video: User-defined Function Constraints
Video: Lab - Serverless Data Analysis (Java/Python) : Part 3
Reading: Codelab - Advanced Queries
Video: Lab 1c Review
Video: How Do You Optimize Queries?
Video: Don't Project Unnecessary Columns
Video: Filter Early and Often Using WHERE Clauses
Video: Do the Biggest Joins First
Video: Low Cardinality GROUP Bys Are Faster
Video: Built-in Functions Are Faster Than JavaScript UDFs
Video: ORDER On the Outermost Query
Video: Wildcard Tables - Standard SQL
Video: Table Partitioning
Video: Example of Table Partitioning
Video: Understand Query Performance
Video: BigQuery Explanation Plans
Video: Understand BigQuery Plans
Video: Monitor Big Query with Stackdriver
Video: Three Categories of BigQuery Pricing and Resources

Graded: Module 1 Quiz

Module 2: Autoscaling Data Processing Pipelines with Dataflow

36 videos, 4 readings

Video: Module 2 Agenda
Video: Elastic Data Processing Pipeline
Video: Open-source API, Google infrastructure
Video: Same code does real-time and batch
Video: Dataflow does ingest, transform and load
Video: Data pipeline + Lab
Video: Dataflow terms and concepts
Video: Python API conceptually similar
Video: Apply Transform to PCollection
Video: Apply Transform to PCollection (Python)
Video: Ingesting data into a pipeline
Video: Can write data out to same formats
Video: Executing pipeline (Java)
Video: Executing pipeline (Python)
Video: Lab - Serverless Data Analysis
Reading: Codelab - A Simple Dataflow Pipeline
Video: Lab 4 Review
Video: MapReduce in Dataflow + Lab
Video: ParDo allows for parallel processing
Video: Python: Map vs. FlatMap
Video: GroupBy operation is akin to shuffle
Video: Combine lets you aggregate
Video: GroupBy and Combine in Python
Video: Combine lets you aggregate
Video: Prefer Combine over GroupByKey
Video: Lab - Serverless Data Analysis (Java/Python) : Part 5
Reading: Codelab - MapReduce in Dataflow
Video: Lab 5 Review
Video: Side Inputs and In-memory Objects
Video: Lab - Serverless Data Analysis (Java/Python) : Part 6
Reading: Codelab - Side inputs and BigQuery
Video: Lab 6 Review
Video: Lab 6 Review Part 2
Video: Streaming and Inputs
Video: Use windows to specify how to aggregate unbounded collections
Video: Lab - Serverless Data Analysis (Java/Python) : Part 7
Reading: Codelab - Streaming into BigQuery
Video: Lab 7 Review
Video: Google Cloud Reference Architecture
Video: Resources

Graded: Module 2 Quiz

FAQs

How It Works

Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from Your Peers

Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.

Certificates

Earn official recognition for your work, and share your success with friends, colleagues, and employers.

Creators

Google Cloud

We help millions of organizations empower their employees, serve their customers, and build what’s next for their businesses with innovative technology created in—and for—the cloud. Our products are engineered for security, reliability, and scalability, running the full stack from infrastructure to applications to devices and hardware. Our teams are dedicated to helping customers apply our technologies to create success.

Pricing

	$99 USD/month	No Certificate
Access to course materials	Available
Access to graded materials	Available
Receive a final grade	Available
Earn a shareable Course Certificate	Available

Ratings and Reviews

Rated 4.4 out of 5 of 27 ratings

Good Intro to google's big query and dataflow.

Assignments were codelabs that can be found in Google's codelabs site

Thank you for the course. I wish the last 3 labs about dataflow were using python code and not java. In general all the material about dataflow is the hardest so far in this specialisation. Not easy to grasp for a person with limited coding skills.

Good Overview

loved it

Share