Azure Data Factory TrainingContinuous IntegrationDelivery
What you’ll learn
How To Create Pipelines In Azure Data Factory.
How To Integrate Azure Data Factory With Azure DevOps.
How To Integrate Azure Data Factory With GitHub.
Continuous Integration and Delivery (CI/CD) in Azure Data Factory.
Learn To Use Pipelines and activities in Azure Data Factory.
How To Create Data set & Linked services in Azure Data Factory.
Transform Data With Mapping Data Flows.
Prepare Data With Wrangling Data Flows.
Learn To Use Different Data Flow activities in Azure Data Factory.
Parameterizations Of Data set and pipelines.
Triggers In Azure Data Factory.
How to build Custom Email Notification In Azure Data Factory.
Introduction to Cosmos DB.
Introduction to Azure Synapse Analytics.
Introduction To Azure SQL.
Dynamic Data Masking in Azure SQL.
Learning Materials For DP 203 Exam
Introductions To Power BI
Requirements
Laptop / Mobil Phone (With headphone)
Internet Connection
Description
Azure Data Factory Masterclass: Azure Data Factory is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores. You can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, Azure Databricks, and Azure SQL Database.Additionally, you can publish your transformed data to data stores such as Azure SQL Data Warehouse for business intelligence (BI) applications to consume. Ultimately, through Azure Data Factory, raw data can be organized into meaningful data stores and data lakes for better business decisions.Mastering Power BI:New addition, We have add ne new module to explain power BI dashboard development. In this course, you are going to learn How to Create Free Azure Subscriptions.Why we need the Azure Data Factory.What are the Key Components Of Azure Data Factory? How to create Azure Data Factory Instances, How to create Azure SQL databases How to create tables In Azure SQL databases and insert data into your tables using SQL Server management studio.How to create a Blob Storage Account using the Azure portal. How to create Azure Data Factory Instances.How to create linked services in Azure Data factory.How to create data set in Azure Data factory.Master different types of activities inside Azure Data Factory.Azure Data Factory Custom Email Notifications.Learn to transform your data with mapping Data FlowLearn to prepare your data with Wrangling Data Flow.Github integrations.Devops integrations.Parameterization in Azure Data Factory Data Set and Pipelines.Continuous Integration and Delivery (CI/CD) in Azure Data Factory.Custom email notification in Azure Data factory (Added in Nov 2020) Slowly changing dimension. (Added in Nov 2020) Incremental Load in Azure Data factory (Added in Nov 2020).Azure Cosmos DB (Added in Dec 2020)Azure Synapse Analytics (Added in Dec 2020)Dynamic Data masking (Added in Dec 2020)Mastering Power BI (Added in Oct 2021)Additional Learning Materials -For DP 203 Exam Preparation.) (Added in Dec 2021)Azure Mapping Data Flow New Module (Added In Dec 2022)
Overview
Section 1: Introduction
Lecture 1 What We Will Learn From This Course ?
Lecture 2 Download Full Resources And Code Used In This Course
Section 2: Azure Basics
Lecture 3 Create Azure Account With Credit Card
Lecture 4 How To Use Azure Account — After Free Trial ,Without Spending Money
Lecture 5 ETL Vs ELT
Lecture 6 Key components Of ADF & SSIS
Section 3: Demo Lab 1: Introductions To Azure Data Factory
Lecture 7 Introduction To Lab 1
Lecture 8 Create Azure SQL Database
Lecture 9 Access Azure SQL Databases Using Azure Query Editor & SSMS
Lecture 10 Connect To Azure SQL Data Base From On-Premise Using SSMS & Create New Tables
Lecture 11 Create A Storage Account
Lecture 12 Connect To Storage Account Using Azure Storage Explore
Lecture 13 Create Azure Data Factory Instance In Azure
Lecture 14 Azure Data Factory For The First Time
Lecture 15 Create Linked Service (Connection To Source & Destinations)
Lecture 16 Create Data Set For Source Tables And Destinations Files
Lecture 17 Create Pipeline And Configure Copy Activity (Azure SQL DB To Storage Account)
Lecture 18 Execute Our Pipeline And Monitor Our Pipeline Executions
Lecture 19 Clean Your Azure Resources
Section 4: Practice Azure (Task 1 & Solution: New Version)
Lecture 20 Task 1: Check Your Understanding
Lecture 21 Create New Resource Group Called ADF
Lecture 22 Create A New Storage Account In Azure
Lecture 23 What is Azure Data Lake V2
Lecture 24 Create Date Lake Storage in Azure
Lecture 25 Create Azure SQL Data Base & Tables For This Demo Part 1
Lecture 26 Create Tables and Insert Data into Azure SQL DB
Lecture 27 Create Azure Data Factory Instances & Linked Services
Lecture 28 Create Multiple Date Set ( To Access Azure SQL, Storage Account & Data Lake)
Lecture 29 Create A Pipeline With Multiple Copy Activities
Lecture 30 Clean Your Resources
Section 5: Parameters In Azure Data Factory
Lecture 31 Use Parameter In your Data Set To Create A Dynamic Pipelines-Part 1
Lecture 32 Use Parameter In your Data Set To Create A Dynamic Pipelines-Part 2
Lecture 33 Use Of Variables In Azure Data Factory
Lecture 34 Use Of System Variables In Azure Data Factory
Lecture 35 Use Set Variable Activity In Azure Data Factory
Section 6: Azure Data Factory Administrations
Lecture 36 Annotations in Azure Data Factory
Lecture 37 User Properties In Azure Data Factory
Section 7: Learn Different Control Flow Activities In ADF Part 1
Lecture 38 Understanding Our Requirements
Lecture 39 Use Of Get Metadata Activity
Lecture 40 Use Wait Activity To Delay Our Process
Lecture 41 Use Of ForEach Activity
Lecture 42 Use Of If Condition Activity
Lecture 43 Use Of Delete Activity
Lecture 44 Assignment
Lecture 45 Demo: Delete Specific Size Files Form Your Storage Account
Section 8: Learn Different Control Flow Activities In ADF
Lecture 46 Use Get Metadata Activity With Azure SQL Server
Lecture 47 Lookup Activity In Azure Data Factory
Lecture 48 ForEach Activity In Azure Data Factory
Lecture 49 ForEach Activity In Azure Data Factory (Copy Data Dynamically With Lookup)
Lecture 50 Until Activity In Azure Data Factory
Lecture 51 Wait Execution Until A Specific Time
Lecture 52 Wait Until A file Exist (Demo For If Condition Activity & Set Variables)
Section 9: Real World Project & Use Cases
Lecture 53 Incrementally copy new and changed files based on Last Modified Date
Lecture 54 Difference Between Incremental Load and Full Load (Animation)
Lecture 55 How We Could Implement Incremental Load In ADF (Single Table – Insert Only)
Lecture 56 Create Infrastructure For Incremental Load Demo
Lecture 57 Create Source Tables and Destination Tables
Lecture 58 Create Dataset And Linked services
Lecture 59 Create A Full Load Azure Data Factory
Lecture 60 Implement A Incremental Load In Azure Data Factory
Lecture 61 What is SCD / Slowly changing dimension?
Lecture 62 How To Implement SCD Type 1 In Azure Data Factory?
Lecture 63 Demo: SCD Type 1
Section 10: Azure Data Factory Custom Email Notifications
Lecture 64 Introduction To Azure Logic Apps
Lecture 65 Lets Create A Logic Apps To Send A Email
Lecture 66 Create Resources & Data Set
Lecture 67 Invoke Azure Logic Apps Using Azure Data Factory Web Activity
Lecture 68 Pass Custom Message From ADF To Azure Logic App
Lecture 69 Pass System Parameter Values From ADF To Azure Logic Apps
Lecture 70 Configure Azure Data Factory To Send A Failed email notification -Last Activity
Lecture 71 Configure Azure Data Factory To Send A Failed email notification -Any Activity
Section 11: Self Hosted Integration Runtime & Data Set Parameters (Lab 3 New Version )
Lecture 72 Introduction To Lab 3
Lecture 73 Create Azure Resources For This Demo
Lecture 74 Create Self Hosted Integration Run Time To Access On-Premise Devices
Lecture 75 Perform Copy Activity Without Parameter
Lecture 76 Create Pipelines With Lookup Activity
Lecture 77 Create Pipelines With Lookup & For Each Activity
Section 12: Mapping Data Flow Demo Part 1 (New Module Added On Dec 2022)
Lecture 78 Introductions to Mapping Data Flow
Lecture 79 How To Use Data Flow UI Part 1
Lecture 80 How To Use Data Flow UI Part 2
Lecture 81 How To Use Data Flow UI Part 3
Lecture 82 Introduction’s To Required Transformations
Lecture 83 Source Transformations: Customer
Lecture 84 Schema drift in Source Transformations
Lecture 85 Source Transformations In Mapping Data Flow: Payments
Lecture 86 Source Transformations: Orders
Lecture 87 Filter Transformations: Restrict Payments Data
Lecture 88 Assignment: Filter Transformations To Restrict Customer Data
Lecture 89 Solutions: Filter Transformations To Restrict Customer Data
Lecture 90 Select Transformations: Select Relevant Columns From Customer Table
Lecture 91 Assignment: Select Relevant Columns From Payments Table
Lecture 92 Solution : Select Relevant Columns From Payments Table
Lecture 93 Select Relevant Columns From Orders Table
Lecture 94 Join Transformations: Join Customers & Orders
Lecture 95 Assignment: Use Join Transformations To Join Customers-Orders With Payments
Lecture 96 Solutions : Use Join Transformations To Join Customers-Orders With Payments
Lecture 97 Remove unwanted columns using Select Transformations
Lecture 98 Use Pivot in Mapping Data Flow
Lecture 99 Sink Transformations In Mapping Data Flow
Lecture 100 Execute Our Data Flow Using Data Flow Activity
Lecture 101 Use Parameters To Make Your Data Flow A Dynamic
Section 13: Mapping Data Flow Demo Part 2
Lecture 102 New Requirements
Lecture 103 Use Inline Data Set To Read Data Using Source Transformations
Lecture 104 Assignment Union Transformations To Combine Data From Sales NA & Sales SA
Lecture 105 Solutions Union Transformations To Combine Data From Sales Files
Lecture 106 UnPivot Transformations In Azure Data Factory Mapping Data Flow
Lecture 107 Derive New Column Transformations
Lecture 108 Derive New Column Transformations
Lecture 109 Remove Unwanted Columns Using Select Transformations
Lecture 110 Introductions To Lookup Transformations
Lecture 111 Join Employee Details And Sales Details Using Join Transformations
Lecture 112 Summarize Your Data Using Aggregate Transformations
Lecture 113 New Branch Transformations
Lecture 114 Conditional Split Transformation
Lecture 115 Demo Conditional Split Transformations In Mapping Data Flow For Multiple Output
Lecture 116 Use Sink Transformations With Inline Data Set
Lecture 117 Executer Employee Sales Details Mapping Data Flow
Section 14: CDC Options in Azure Data Factory Mapping Data Flow
Lecture 118 Introductions To CDC in Mapping Data Flow
Lecture 119 Create Source And Sink For CDC Demo
Lecture 120 Enable CDC at Source – Azure SQL Data Base
Lecture 121 Configure Azure Data Factory Mapping Data Flow To Use Change Data Capture
Lecture 122 Test Change Data Capture Using ADF Mapping Data Flow
Lecture 123 Important: Disable CDC and Change Azure SQL Server Configurations
Section 15: Real Time Scenario Based Tasks in Azure Data Factory
Lecture 124 Read Data From Github : Copy Data From An HTTP Endpoint
Lecture 125 Problem And Solutions
Lecture 126 Use Pipeline Variables
Lecture 127 Use Of Pipeline Parameter
Lecture 128 Use Lookup Activity And For Each Activity
Lecture 129 Final Solutions : Lookup ForEach & Copy Activity
Section 16: Introduction To GitHub
Lecture 130 What We Will Learn From This Module
Lecture 131 How To Create GitHub Account
Lecture 132 How To Create Repository And Files In Your GitHub
Lecture 133 How To Create New Branch
Lecture 134 How To Merge Code Changes From GitHub Feature Branch To Master Branch
Lecture 135 How To Delete A Repository From Your GitHub Account
Section 17: Integrate Azure Data Factory With GitHub
Lecture 136 Create A GitHub Repository (Folder) For Azure Data Factory
Lecture 137 Integrate GitHub Repository With Azure Data Factory
Lecture 138 Publish Your changes From Git To Azure Data Factory
Lecture 139 Create Different Feature Branches For New Development & Code Changes
Section 18: Integrate Azure Data Factory With Azure DevOps
Lecture 140 Create DevOps Account
Lecture 141 Create Project And Repository In DevOps And Integrate ADF With Azure Devops
Lecture 142 How To Use Master Branch (Collaboration Branch For Publishing Changes To ADF)
Lecture 143 How To Use Devops Feature Branches and Pull Request To Marge your Code Changes
Section 19: Access Azure Storage Account Data In A Secure Way (With Azure Key Vault)
Lecture 144 How To Use Azure Key Vault To Access Data By Using Connections String
Lecture 145 How To Rotate Keys To Control Your Data Access.
Section 20: Continuous Integration & Delivery (CI/CD) in Azure Data Factory
Lecture 146 Introduction To Continuous Integration & Delivery In Azure Data Factory (CI/CD)
Lecture 147 Create Different Resource Groups For Different Environments
Lecture 148 Create Three Different Storage Account (Development / Production & Test)
Lecture 149 Configure Input & Output Containers For Different Environment
Lecture 150 Create Key Vault To Access Your Date in Secure Way
Lecture 151 Deploy Different Azure Data Factory Instances In Different Environment
Lecture 152 Grant Azure Data Factory To Access Key Vault Secrete (For Three Environment)
Lecture 153 Store Your Storage Account Connection Details Inside Azure Key Vault Secrete
Lecture 154 Create A Demo Pipeline & Test Our Application Execution
Lecture 155 Integrate Azure Data Factory With GitHub Use Feature Branch/Pull Request & Merge
Lecture 156 Create A DevOps Project & Configure Artifacts As GitHub Inside Your New Release
Lecture 157 Create Our First Release To Test Environment
Lecture 158 Use Variable & Variable Group in A Pipeline Release ( For Dynamic Deployment)
Lecture 159 Create Second Release To Test and Production By Using Our Environment Variable
Lecture 160 Automate Our Release Based On ADF Publishing
Section 21: Triggers In Azure Data Factory (From DP-200 Exam Preparation Course)
Lecture 161 Notes
Lecture 162 Azure Data Factory Triggers :: Introductions
Lecture 163 Scheduled Triggers In Azure Data Factory
Lecture 164 Tumbling Window Trigger in Azure Data Factory
Lecture 165 Trigger Azure Data Factory Based On Blob Event
Lecture 166 How To Use Event Trigger With Parameter
Lecture 167 How To Create Dependent Trigger With Tumbling Window Trigger
Section 22: Mastering Power BI
Lecture 168 Introduction To Power BI Project
Lecture 169 Install Power BI
Lecture 170 Data Source
Lecture 171 Import Data Into Power BI
Lecture 172 Fix The Column Issue Using Power Query / Transform Data
Lecture 173 Data Preparation: Deriving New Column/Data Using Add Columns
Lecture 174 Data Preparation: Basic Data Cleaning Using Power BI
Lecture 175 Power BI Visualization: Add Slicers
Lecture 176 Power BI Visualization: Add Cards
Lecture 177 Power BI Visualization: Create a Power BI Pie Chart
Lecture 178 Power BI Visualization: Create A Power BI Bar and column charts
Lecture 179 Power BI Visualization : Map-Based Visualization in Power BI
Section 23: Introduction To NoSql & Cosmos DB- How To Access Cosmos From ADF
Lecture 180 An Introduction To Azure Cosmos DB
Lecture 181 Create an Azure Cosmos DB Database And Save Data
Lecture 182 Execute Query (Query data in your Azure Cosmos DB database)
Lecture 183 Copy Data From Azure SQL To Cosmos DB database Using Azure Data Factory
Section 24: Introduction To SSIS & Azure
Lecture 184 How to Download and Install Visual Studio 2019
Lecture 185 Install SSIS & Azure Feature Pack For SSIS
Lecture 186 Task 1: Upload Data Into Azure Storage Account Using SSIS task
Lecture 187 Task 2: Read Data from Azure Storage Account Using SSIS
Section 25: Azure Synapse Analytics
Lecture 188 Introduction To Azure Synapse Analytics
Lecture 189 Demo (Synapse Analytics Studio)
Lecture 190 Create Azure Synapse Analytics
Lecture 191 Create Azure SQL For Demo 1
Lecture 192 Connect To Azure Synapse SQL Pool From SSMS
Lecture 193 Open Synapse Studio
Lecture 194 Create Dedicated SQL Pool
Lecture 195 Create Tables In Your SQL Pool & Execute SQL Statements
Lecture 196 Create Spark Pool Using Synapse Analytics
Lecture 197 Create Notebook and Database Using Spark Pool
Lecture 198 Create OnDemand SQL pool
Lecture 199 Access External Data source and Create External Tables
Lecture 200 Introduction To Demo (Pipelines In Synapse Analytics)
Lecture 201 Create Pipelines In Azure Synapse Analytics
Section 26: Additional Learning Materials (DP 203 Exam Preparation)
Lecture 202 Dynamic Data Masking
Lecture 203 Demo: Dynamic Data Masking In Azure
Lecture 204 What Is Azure Storage Account?
Lecture 205 Demo: Azure Storage Account Creations And Uses
Lecture 206 How To Use Storage Explore From Portal
Lecture 207 Storage Explore Desktop Version
Lecture 208 Redundancy In Primary And Secondary Region
Lecture 209 Read Access To Data in The Secondary Region
Section 27: Interview Questions
Lecture 210 Question 1
Lecture 211 Question 2
Lecture 212 Question 3
Lecture 213 Question 4
Lecture 214 Question 5
Lecture 215 Question 6
Lecture 216 Question 7
Lecture 217 Question 8
Lecture 218 Question 9
Lecture 219 Question 10
Lecture 220 Question 11
Lecture 221 Question 12
Lecture 222 Question 13
Lecture 223 Question 14
Section 28: Master Different Wrangling Data Flow Transformations in ADF
Lecture 224 How To Use Wrangling Data Flows in Azure Data Factory (Group By Transformations)
Lecture 225 Common Transformations and Actions In Wrangling Data Flow
Lecture 226 Conditional Columns & Other Common Transformations In Wrangling Data Flow
Lecture 227 Merge Transformations:: Wrangling data flows
Section 29: Mapping Data Flow Demo Part 1 (Old Module)
Lecture 228 What Is Mapping data flows In Azure Data Factor 26-12-2022
Lecture 229 Source And Sin in Azure Mapping Data Flow
Lecture 230 Aggregate Transformation In Mapping Data Flow
Lecture 231 Filter Transformations In Mapping Data Flow
Lecture 232 Conditional Split Transformation Demo
Lecture 233 Derived Column Data Transformations
Lecture 234 New Branch Transformations
Lecture 235 Sort Transformations
Lecture 236 Select Transformation
Lecture 237 Use Of Join Transformation & Types Of Join Transformations In ADF
Lecture 238 How To Use Join Transformations In ADF
Lecture 239 Exist And Does not Exist Transformation To Filter Your Data
Section 30: Old Videos-Deleted
Lecture 240 Old videos
Section 31: Bonus
Lecture 241 Bonus
Any students who want to learn Azure Data Factory,Any student who want to implement Different Data Solutions In Azure,Any student who want to learn batch processing system with Azure Data Factory,Any student who want to learn azure synapse analytics
Course Information:
Udemy | English | 19h 56m | 13.19 GB
Created by: Step2C Education
You Can See More Courses in the Developer >> Greetings from CourseDown.com