Azure Data Factory TrainingContinuous IntegrationDelivery
What you’ll learn
How To Create Pipelines In Azure Data Factory.
How To Integrate Azure Data Factory With Azure DevOps.
How To Integrate Azure Data Factory With GitHub.
Continuous Integration and Delivery (CI/CD) in Azure Data Factory.
Learn To Use Pipelines and activities in Azure Data Factory.
How To Create Data set & Linked services in Azure Data Factory.
Transform Data With Mapping Data Flows.
Prepare Data With Wrangling Data Flows.
Learn To Use Different Data Flow activities in Azure Data Factory.
Parameterizations Of Data set and pipelines.
Triggers In Azure Data Factory.
How to build Custom Email Notification In Azure Data Factory.
Introduction to Cosmos DB.
Introduction to Azure Synapse Analytics.
Introduction To Azure SQL.
Dynamic Data Masking in Azure SQL.
Learning Materials For DP 203 Exam
Introductions To Power BI
Requirements
Laptop / Mobil Phone (With headphone)
Internet Connection
Description
Azure Data Factory Masterclass: Azure Data Factory is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores. You can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, Azure Databricks, and Azure SQL Database.Additionally, you can publish your transformed data to data stores such as Azure SQL Data Warehouse for business intelligence (BI) applications to consume. Ultimately, through Azure Data Factory, raw data can be organized into meaningful data stores and data lakes for better business decisions.Mastering Power BI:New addition, We have add ne new module to explain power BI dashboard development. In this course, you are going to learn How to Create Free Azure Subscriptions.Why we need the Azure Data Factory.What are the Key Components Of Azure Data Factory? How to create Azure Data Factory Instances, How to create Azure SQL databases How to create tables In Azure SQL databases and insert data into your tables using SQL Server management studio.How to create a Blob Storage Account using the Azure portal. How to create Azure Data Factory Instances.How to create linked services in Azure Data factory.How to create data set in Azure Data factory.Master different types of activities inside Azure Data Factory.Azure Data Factory Custom Email Notifications.Learn to transform your data with mapping Data FlowLearn to prepare your data with Wrangling Data Flow.Github integrations.Devops integrations.Parameterization in Azure Data Factory Data Set and Pipelines.Continuous Integration and Delivery (CI/CD) in Azure Data Factory.Custom email notification in Azure Data factory (Added in Nov 2020) Slowly changing dimension. (Added in Nov 2020) Incremental Load in Azure Data factory (Added in Nov 2020).Azure Cosmos DB (Added in Dec 2020)Azure Synapse Analytics (Added in Dec 2020)Dynamic Data masking (Added in Dec 2020)Mastering Power BI (Added in Oct 2021)Additional Learning Materials -For DP 203 Exam Preparation.) (Added in Dec 2021)Azure Mapping Data Flow New Module (Added In Dec 2022)
Overview
Section 1: Introduction
Lecture 1 What We Will Learn From This Course ?
Lecture 2 Download Full Resources And Code Used In This Course
Section 2: Azure Basics
Lecture 3 Create Azure Account With Credit Card
Lecture 4 How To Use Azure Account — After Free Trial ,Without Spending Money
Lecture 5 ETL Vs ELT
Lecture 6 Key components Of ADF & SSIS
Section 3: Demo Lab 1: Introductions To Azure Data Factory
Lecture 7 Introduction To Lab 1
Lecture 8 Create Azure SQL Database
Lecture 9 Access Azure SQL Databases Using Azure Query Editor & SSMS
Lecture 10 Connect To Azure SQL Data Base From On-Premise Using SSMS & Create New Tables
Lecture 11 Create A Storage Account
Lecture 12 Connect To Storage Account Using Azure Storage Explore
Lecture 13 Create Azure Data Factory Instance In Azure
Lecture 14 Azure Data Factory For The First Time
Lecture 15 Create Linked Service (Connection To Source & Destinations)
Lecture 16 Create Data Set For Source Tables And Destinations Files
Lecture 17 Create Pipeline And Configure Copy Activity (Azure SQL DB To Storage Account)
Lecture 18 Execute Our Pipeline And Monitor Our Pipeline Executions
Lecture 19 Clean Your Azure Resources
Section 4: Practice Azure (Task 1 & Solution: New Version)
Lecture 20 Task 1: Check Your Understanding
Lecture 21 Create New Resource Group Called ADF
Lecture 22 Create A New Storage Account In Azure
Lecture 23 What is Azure Data Lake V2
Lecture 24 Create Date Lake Storage in Azure
Lecture 25 Create Azure SQL Data Base & Tables For This Demo Part 1
Lecture 26 Create Tables and Insert Data into Azure SQL DB
Lecture 27 Create Azure Data Factory Instances & Linked Services
Lecture 28 Create Multiple Date Set ( To Access Azure SQL, Storage Account & Data Lake)
Lecture 29 Create A Pipeline With Multiple Copy Activities
Lecture 30 Clean Your Resources
Section 5: Parameters In Azure Data Factory
Lecture 31 Use Parameter In your Data Set To Create Dynamic Pipelines
Lecture 32 Parameter in Pipeline & Execute a Parameterized Pipeline With Execute Pipeline
Section 6: Learn Different Control Flow Activities In ADF Part 1
Lecture 33 Understanding Our Requirements
Lecture 34 Use Of Get Metadata Activity
Lecture 35 Use Wait Activity To Delay Our Process
Lecture 36 Use Of ForEach Activity
Lecture 37 Use Of If Condition Activity
Lecture 38 Use Of Delete Activity
Lecture 39 Assignment
Lecture 40 Demo: Delete Specific Size Files Form Your Storage Account
Section 7: Learn Different Control Flow Activities In ADF
Lecture 41 Use Get Metadata Activity With Azure SQL Server
Lecture 42 Lookup Activity In Azure Data Factory
Lecture 43 ForEach Activity In Azure Data Factory
Lecture 44 ForEach Activity In Azure Data Factory (Copy Data Dynamically With Lookup)
Lecture 45 Until Activity In Azure Data Factory
Lecture 46 Wait Execution Until A Specific Time
Lecture 47 Wait Until A file Exist (Demo For If Condition Activity & Set Variables)
Section 8: Real World Project & Use Cases
Lecture 48 Incrementally copy new and changed files based on Last Modified Date
Lecture 49 Difference Between Incremental Load and Full Load (Animation)
Lecture 50 How We Could Implement Incremental Load In ADF (Single Table – Insert Only)
Lecture 51 Create Infrastructure For Incremental Load Demo
Lecture 52 Create Source Tables and Destination Tables
Lecture 53 Create Dataset And Linked services
Lecture 54 Create A Full Load Azure Data Factory
Lecture 55 Implement A Incremental Load In Azure Data Factory
Lecture 56 What is SCD / Slowly changing dimension?
Lecture 57 How To Implement SCD Type 1 In Azure Data Factory?
Lecture 58 Demo: SCD Type 1
Section 9: Azure Data Factory Custom Email Notifications
Lecture 59 Introduction To Azure Logic Apps
Lecture 60 Lets Create A Logic Apps To Send A Email
Lecture 61 Create Resources & Data Set
Lecture 62 Invoke Azure Logic Apps Using Azure Data Factory Web Activity
Lecture 63 Pass Custom Message From ADF To Azure Logic App
Lecture 64 Pass System Parameter Values From ADF To Azure Logic Apps
Lecture 65 Configure Azure Data Factory To Send A Failed email notification -Last Activity
Lecture 66 Configure Azure Data Factory To Send A Failed email notification -Any Activity
Section 10: Self Hosted Integration Runtime & Data Set Parameters (Lab 3 New Version )
Lecture 67 Introduction To Lab 3
Lecture 68 Create Azure Resources For This Demo
Lecture 69 Create Self Hosted Integration Run Time To Access On-Premise Devices
Lecture 70 Perform Copy Activity Without Parameter
Lecture 71 Create Pipelines With Lookup Activity
Lecture 72 Create Pipelines With Lookup & For Each Activity
Section 11: Mapping Data Flow Demo Part 1 (New Module Added On Dec 2022)
Lecture 73 Introductions to Mapping Data Flow
Lecture 74 How To Use Data Flow UI Part 1
Lecture 75 How To Use Data Flow UI Part 2
Lecture 76 How To Use Data Flow UI Part 3
Lecture 77 Introduction’s To Required Transformations
Lecture 78 Source Transformations: Customer
Lecture 79 Schema drift in Source Transformations
Lecture 80 Source Transformations In Mapping Data Flow: Payments
Lecture 81 Source Transformations: Orders
Lecture 82 Filter Transformations: Restrict Payments Data
Lecture 83 Assignment: Filter Transformations To Restrict Customer Data
Lecture 84 Solutions: Filter Transformations To Restrict Customer Data
Lecture 85 Select Transformations: Select Relevant Columns From Customer Table
Lecture 86 Assignment: Select Relevant Columns From Payments Table
Lecture 87 Solution : Select Relevant Columns From Payments Table
Lecture 88 Select Relevant Columns From Orders Table
Lecture 89 Join Transformations: Join Customers & Orders
Lecture 90 Assignment: Use Join Transformations To Join Customers-Orders With Payments
Lecture 91 Solutions : Use Join Transformations To Join Customers-Orders With Payments
Lecture 92 Remove unwanted columns using Select Transformations
Lecture 93 Use Pivot in Mapping Data Flow
Lecture 94 Sink Transformations In Mapping Data Flow
Lecture 95 Execute Our Data Flow Using Data Flow Activity
Lecture 96 Use Parameters To Make Your Data Flow A Dynamic
Section 12: Mapping Data Flow Demo Part 2
Lecture 97 New Requirements
Lecture 98 Use Inline Data Set To Read Data Using Source Transformations
Lecture 99 Assignment Union Transformations To Combine Data From Sales NA & Sales SA
Lecture 100 Solutions Union Transformations To Combine Data From Sales Files
Lecture 101 UnPivot Transformations In Azure Data Factory Mapping Data Flow
Lecture 102 Derive New Column Transformations
Lecture 103 Derive New Column Transformations
Lecture 104 Remove Unwanted Columns Using Select Transformations
Lecture 105 Introductions To Lookup Transformations
Lecture 106 Join Employee Details And Sales Details Using Join Transformations
Lecture 107 Summarize Your Data Using Aggregate Transformations
Lecture 108 New Branch Transformations
Lecture 109 Conditional Split Transformation
Lecture 110 Demo Conditional Split Transformations In Mapping Data Flow For Multiple Output
Lecture 111 Use Sink Transformations With Inline Data Set
Lecture 112 Executer Employee Sales Details Mapping Data Flow
Section 13: Real Time Scenario Based Tasks in Azure Data Factory
Lecture 113 Read Data From Github : Copy Data From An HTTP Endpoint
Lecture 114 Problem And Solutions
Lecture 115 Use Pipeline Variables
Lecture 116 Use Of Pipeline Parameter
Lecture 117 Use Lookup Activity And For Each Activity
Lecture 118 Final Solutions : Lookup ForEach & Copy Activity
Section 14: Introduction To GitHub
Lecture 119 What We Will Learn From This Module
Lecture 120 How To Create GitHub Account
Lecture 121 How To Create Repository And Files In Your GitHub
Lecture 122 How To Create New Branch
Lecture 123 How To Merge Code Changes From GitHub Feature Branch To Master Branch
Lecture 124 How To Delete A Repository From Your GitHub Account
Section 15: Integrate Azure Data Factory With GitHub
Lecture 125 Create A GitHub Repository (Folder) For Azure Data Factory
Lecture 126 Integrate GitHub Repository With Azure Data Factory
Lecture 127 Publish Your changes From Git To Azure Data Factory
Lecture 128 Create Different Feature Branches For New Development & Code Changes
Section 16: Integrate Azure Data Factory With Azure DevOps
Lecture 129 Create DevOps Account
Lecture 130 Create Project And Repository In DevOps And Integrate ADF With Azure Devops
Lecture 131 How To Use Master Branch (Collaboration Branch For Publishing Changes To ADF)
Lecture 132 How To Use Devops Feature Branches and Pull Request To Marge your Code Changes
Section 17: Access Azure Storage Account Data In A Secure Way (With Azure Key Vault)
Lecture 133 How To Use Azure Key Vault To Access Data By Using Connections String
Lecture 134 How To Rotate Keys To Control Your Data Access.
Section 18: Continuous Integration & Delivery (CI/CD) in Azure Data Factory
Lecture 135 Introduction To Continuous Integration & Delivery In Azure Data Factory (CI/CD)
Lecture 136 Create Different Resource Groups For Different Environments
Lecture 137 Create Three Different Storage Account (Development / Production & Test)
Lecture 138 Configure Input & Output Containers For Different Environment
Lecture 139 Create Key Vault To Access Your Date in Secure Way
Lecture 140 Deploy Different Azure Data Factory Instances In Different Environment
Lecture 141 Grant Azure Data Factory To Access Key Vault Secrete (For Three Environment)
Lecture 142 Store Your Storage Account Connection Details Inside Azure Key Vault Secrete
Lecture 143 Create A Demo Pipeline & Test Our Application Execution
Lecture 144 Integrate Azure Data Factory With GitHub Use Feature Branch/Pull Request & Merge
Lecture 145 Create A DevOps Project & Configure Artifacts As GitHub Inside Your New Release
Lecture 146 Create Our First Release To Test Environment
Lecture 147 Use Variable & Variable Group in A Pipeline Release ( For Dynamic Deployment)
Lecture 148 Create Second Release To Test and Production By Using Our Environment Variable
Lecture 149 Automate Our Release Based On ADF Publishing
Section 19: Triggers In Azure Data Factory (From DP-200 Exam Preparation Course)
Lecture 150 Notes
Lecture 151 Azure Data Factory Triggers :: Introductions
Lecture 152 Scheduled Triggers In Azure Data Factory
Lecture 153 Tumbling Window Trigger in Azure Data Factory
Lecture 154 Trigger Azure Data Factory Based On Blob Event
Lecture 155 How To Use Event Trigger With Parameter
Lecture 156 How To Create Dependent Trigger With Tumbling Window Trigger
Section 20: Mastering Power BI
Lecture 157 Introduction To Power BI Project
Lecture 158 Install Power BI
Lecture 159 Data Source
Lecture 160 Import Data Into Power BI
Lecture 161 Fix The Column Issue Using Power Query / Transform Data
Lecture 162 Data Preparation: Deriving New Column/Data Using Add Columns
Lecture 163 Data Preparation: Basic Data Cleaning Using Power BI
Lecture 164 Power BI Visualization: Add Slicers
Lecture 165 Power BI Visualization: Add Cards
Lecture 166 Power BI Visualization: Create a Power BI Pie Chart
Lecture 167 Power BI Visualization: Create A Power BI Bar and column charts
Lecture 168 Power BI Visualization : Map-Based Visualization in Power BI
Section 21: Introduction To NoSql & Cosmos DB- How To Access Cosmos From ADF
Lecture 169 An Introduction To Azure Cosmos DB
Lecture 170 Create an Azure Cosmos DB Database And Save Data
Lecture 171 Execute Query (Query data in your Azure Cosmos DB database)
Lecture 172 Copy Data From Azure SQL To Cosmos DB database Using Azure Data Factory
Section 22: Introduction To SSIS & Azure
Lecture 173 How to Download and Install Visual Studio 2019
Lecture 174 Install SSIS & Azure Feature Pack For SSIS
Lecture 175 Task 1: Upload Data Into Azure Storage Account Using SSIS task
Lecture 176 Task 2: Read Data from Azure Storage Account Using SSIS
Section 23: Azure Synapse Analytics
Lecture 177 Introduction To Azure Synapse Analytics
Lecture 178 Demo (Synapse Analytics Studio)
Lecture 179 Create Azure Synapse Analytics
Lecture 180 Create Azure SQL For Demo 1
Lecture 181 Connect To Azure Synapse SQL Pool From SSMS
Lecture 182 Open Synapse Studio
Lecture 183 Create Dedicated SQL Pool
Lecture 184 Create Tables In Your SQL Pool & Execute SQL Statements
Lecture 185 Create Spark Pool Using Synapse Analytics
Lecture 186 Create Notebook and Database Using Spark Pool
Lecture 187 Create OnDemand SQL pool
Lecture 188 Access External Data source and Create External Tables
Lecture 189 Introduction To Demo (Pipelines In Synapse Analytics)
Lecture 190 Create Pipelines In Azure Synapse Analytics
Section 24: Additional Learning Materials (DP 203 Exam Preparation)
Lecture 191 Dynamic Data Masking
Lecture 192 Demo: Dynamic Data Masking In Azure
Lecture 193 What Is Azure Storage Account?
Lecture 194 Demo: Azure Storage Account Creations And Uses
Lecture 195 How To Use Storage Explore From Portal
Lecture 196 Storage Explore Desktop Version
Lecture 197 Redundancy In Primary And Secondary Region
Lecture 198 Read Access To Data in The Secondary Region
Section 25: Interview Questions
Lecture 199 Question 1
Lecture 200 Question 2
Lecture 201 Question 3
Lecture 202 Question 4
Lecture 203 Question 5
Lecture 204 Question 6
Lecture 205 Question 7
Lecture 206 Question 8
Lecture 207 Question 9
Section 26: Master Different Wrangling Data Flow Transformations in ADF
Lecture 208 How To Use Wrangling Data Flows in Azure Data Factory (Group By Transformations)
Lecture 209 Common Transformations and Actions In Wrangling Data Flow
Lecture 210 Conditional Columns & Other Common Transformations In Wrangling Data Flow
Lecture 211 Merge Transformations:: Wrangling data flows
Section 27: Mapping Data Flow Demo Part 1 (Old Module)
Lecture 212 What Is Mapping data flows In Azure Data Factor 26-12-2022
Lecture 213 Source And Sin in Azure Mapping Data Flow
Lecture 214 Aggregate Transformation In Mapping Data Flow
Lecture 215 Filter Transformations In Mapping Data Flow
Lecture 216 Conditional Split Transformation Demo
Lecture 217 Derived Column Data Transformations
Lecture 218 New Branch Transformations
Lecture 219 Sort Transformations
Lecture 220 Select Transformation
Lecture 221 Use Of Join Transformation & Types Of Join Transformations In ADF
Lecture 222 How To Use Join Transformations In ADF
Lecture 223 Exist And Does not Exist Transformation To Filter Your Data
Section 28: Old Videos-Deleted
Lecture 224 Old videos
Section 29: Bonus
Lecture 225 Bonus
Any students who want to learn Azure Data Factory,Any student who want to implement Different Data Solutions In Azure,Any student who want to learn batch processing system with Azure Data Factory,Any student who want to learn azure synapse analytics
Course Information:
Udemy | English | 18h 48m | 11.82 GB
Created by: Step2C Education
You Can See More Courses in the Developer >> Greetings from CourseDown.com