DP-601T00 - Implementing a Lakehouse with Microsoft Fabric

Course Description

This course is designed to build your foundational skills in data engineering on Microsoft Fabric, focusing on the Lakehouse concept. This course will explore the powerful capabilities of Apache Spark for distributed data processing and the essential techniques for efficient data management, versioning and reliability by working with Delta Lake tables. This course will also explore data ingestion and orchestration using Dataflows Gen2 and Data Factory pipelines. This course includes a combination of lectures and hands-on exercises that will prepare you to work with lakehouses in Microsoft Fabric.
1 Days
Contact us for pricing
 

Who should attend

The primary audience for this course is data professionals who are familiar with data modeling, extraction and analytics. It is designed for professionals who are interested in gaining knowledge about Lakehouse architecture, the Microsoft Fabric platform and how to enable end-to-end analytics using these technologies.

Module 1 - Introduction to end-to-end analytics using Microsoft Fabric

Microsoft Fabric components and workloads
OneLake and unified data storage
Data engineer, analyst and scientist responsibilities
Fabric capacities, workspaces and items
Typical end-to-end analytics workflow

Module 2 - Get started with lakehouses in Microsoft Fabric

Lakehouse concepts and use cases
Files area versus managed tables
Delta tables and the SQL analytics endpoint
Data ingestion options
Querying lakehouse data with T-SQL

Exercise - Create a Microsoft Fabric lakehouse

Module 3 - Use Apache Spark in Microsoft Fabric

Spark pools and session configuration
Fabric notebooks
Loading data into a DataFrame
Selecting, filtering, grouping and transforming data
Spark SQL and temporary views
Notebook visualisations
Choosing between Spark and SQL

Exercise - Analyze data with Apache Spark

Module 4 - Work with Delta Lake tables in Microsoft Fabric

Delta Lake transaction architecture
Managed and external tables
Creating and writing Delta tables
Append, overwrite and merge patterns
Table optimisation and maintenance
Table history and versioning
Structured streaming concepts

Exercise - Use delta tables in Apache Spark

Module 5 - Ingest data with Dataflows Gen2 in Microsoft Fabric

Dataflows Gen2 architecture
Power Query transformations
Sources, credentials and destinations
Data type and data-quality handling
Loading data into a lakehouse
Refresh and pipeline integration

Exercise - Create and use a Dataflow Gen2 in Microsoft Fabric

Module 6 - Orchestrate processes and data movement with Microsoft Fabric

Fabric Data Factory pipelines
Copy Data assistant and Copy Data activity
Notebook and Dataflow activities
Parameters and dynamic expressions
Dependencies and failure paths
Scheduling, running and monitoring pipelines

Exercise - Ingest data with a pipeline

Module 7 - Organize a Fabric lakehouse using medallion architecture design

Purpose of Bronze, Silver and Gold layers
Raw-data preservation
Cleansing, deduplication and conformance
Business-ready and dimensional data
Partitioning and performance
Security and governance considerations
Serving data to Power BI

Exercise - Organize your Fabric lakehouse using a medallion architecture
Azure SQLAzureApache SparkLakehouseFabricMicrosoft FabricDelta LakeDataflows Gen2Data FactoryDP-601DP601