Building Realtime Pipelines in Cloud Data Fusion

About this course

This is a self-paced lab that takes place in the Google Cloud console. In addition to batch pipelines, Data Fusion also allows you to create real-time pipelines, that can process events as they are generated. Currently, realtime pipelines execute using Apache Spark Streaming on Cloud Dataproc clusters. In this lab, you will learn how to build a streaming pipeline using Data Fusion.

What you will learn

• Learn how to create a realtime pipeline
• Learn how to configure the Pub/Sub source plugin in Data Fusion
• Learn how to use Wrangler to define transformations for data located in unsupported connections.