Snowflake Masterclassstored procdemosBest practicesLabs
What you’ll learn
Snowflake architecture in detail.
How data processing happens in snowflake.
Secure data sharing in snowflake.
Understand different caching layers in snowflake.
Continues data load using snowpipe.
Clustering tables in snowflake and how it works under the hood.
Load data and unload data from AWS , GOOGLE and AZURE cloud
Loading unstructured data.
Snowflake internal stage and external stages.
Snowflake data sampling and its importance.
Detail overview about, snowflake clone , failsafe and data retention.
Unique features of Snowfalke ( time travel , undrop , table swap)
Scheduling using Tasks
Streams in snowflake
Materialized views
Best practices in snowflake
Snowpro certification compatible.
Snowflake stored procedures.
Data masking
Snowsight and warehosue utilization dashboard.
Snowflake access control.
Requirements
Familiarity with SQL language and basic knowledge on database.
Description
If you would like to understand how to build workflows using Airflow, Talend and Snowflake Please check out my new course,Snowflake cloud database with ETL(Airflow+Python+Talend)Latest update : Added new section on snowflake external tables. Added new section on ROW ACCESS POLICY in snowflake. Added new section on Tagging in Snowflake This course is the result of my over 4 years of experience working with Snowflake. Snowflake is a fantastic tool to handle huge volumes of data. You can load, query and export, and run transactional queries on TB’s and PB’s of data easily.You don’t have to worry about infrastructure setup, security, performance, tuning, Garbage collection, node failures etc etc… if you have already worked with Spark with EMR or with Data bricks you will be amazed by snowflake performance and simplicity.But there is a catch, if you are not fully aware of how things works in snowflake, you will easily loose control and end up writing costly queries and end up using many of snowflake features wrongly.I mentioned costly queries because snowflake is build for performance.Costly queries and wrong design decisions in snowflake will sky rocket your snowflake bill. I have made lot of such mistakes and learned from those mistakes.This course is an effort to walk you through such experiences so that you don’t end up doing same mistakes which i did!!!Few things can only be learned by experience you can’t find them in documentations this course will expose such details to you.Hope you join me in this journey……………….The course is broadly divided into 3 modules,Module#1 Lectures ————>: 38 Real time Demos—–>: 11 Assignments———>: 10 Quiz——————>: 6 This module will concretize snowflake fundamental concepts and architecture. By the end of this module you will be having very strong foundation about snowflake.Module#2 Lectures ————>: 46 Real time Demos—–>: 31 Assignments———>: 14 Quiz——————>: 2 In this module i will walk you through, different techniques and options to load and unload data in snowflake. By the end of this module, you will be having great confidence in using COPY command and different data processing objects in snowflake.Module#3 Lectures ————>: 45 Real time Demos—–>: 35 Quiz——————>: 3 In this module, we closely evaluate important features of snowflake. This module helps you to understand, how each feature works under the hood. Through this knowledge you will be in a better position to decide, when to use the feature and when not to use the snowflake features.High level overview of all modules with sections details is given below,In module#1, > Snowflake environment setup. > Snowflake architecture. > Snowflake caching , micro-partitions, clusters. > Snowflake clustering deep dive. > Snowflake virtual warehouse layer. > Performance tuning and DB characteristics.In module#2, > Loading data in snowflake. > Loading data using internal stage. > Load data from aws s3. > Load data from google gcs. > Load data from Azure. > Copy command options. > Working with copy command. > Assignments. > Loading unstructured dataIn module#3, > Snowpipe in snowflake. > Data sharing in snowflake. > Time travel. > Fail safe. > Clone feature. > Data sampling. > Tasks in snowflake. > Streaming in snowflake. > Continuous data load. > Materialized views.
Overview
Section 1: High level overview
Lecture 1 Course structure.
Lecture 2 Latest updates !!!!
Section 2: /********Module-1 starts******/
Lecture 3 Module 1 introduction.
Section 3: Snowflake environment setup
Lecture 4 Creating snowflake trail account
Lecture 5 Snowflake web console introduction part1
Lecture 6 Snowflake web console introduction part2
Lecture 7 Web console limitation
Lecture 8 Creating database and table intro to sample data.
Lecture 9 Section revise notes
Section 4: Snowflake architecture
Lecture 10 Snowflake architecture introduction.
Lecture 11 Shared disk architecture.
Lecture 12 Shared nothing architecture part 1
Lecture 13 Shared nothing architecture part 2
Lecture 14 Snowflake architecture revisit.
Lecture 15 The impact.
Lecture 16 Further reading Resources
Section 5: Caching in snowflake data warehouse
Lecture 17 Introduction.
Lecture 18 [Architecture demo] Caching in snowflake.
Lecture 19 [Architecture demo] Caching revisit.
Lecture 20 [Architecture demo] Lessons learned.
Lecture 21 Clarifications
Section 6: Clustering in snowflake.
Lecture 22 introduction.
Lecture 23 Query processing in snowflake.
Lecture 24 Micro partitions in snowflake.
Lecture 25 Micro partitions depth.
Lecture 26 [Demo] Applying clustering brief intro.
Section 7: Clustering — Deep dive.
Lecture 27 Section introduction.
Lecture 28 [Demo] Checking clustering information
Lecture 29 [Demo] How clustering works.
Lecture 30 [Demo] Improve performance without applying clustering
Lecture 31 [Demo] Manual re-clustering
Lecture 32 [Demo] How to choose clustering keys.
Lecture 33 Section Recap.
Lecture 34 Snowflake clustering documentation.
Section 8: Introduction to virtual warehouse creation.
Lecture 35 Introduction to virtual warehouse.
Lecture 36 Virtual warehouse creation.
Lecture 37 Scaling policy
Section 9: Performance tuning and DB characterstics
Lecture 38 What is performance tuning in snowflake.
Lecture 39 How query optimization works.
Lecture 40 Concurrency control.
Lecture 41 [Demo] How update transaction works.
Lecture 42 [Demo] No constraints.
Lecture 43 Q/A
Section 10: Snowsight and warehouse dashboard
Lecture 44 Introduction
Lecture 45 Snowsight_overview_part1
Lecture 46 Snowsight_overview_part2
Lecture 47 Warehouse idle time intro
Lecture 48 Warehouse analogy
Lecture 49 Metadata views
Lecture 50 Calculating idle cost
Lecture 51 Adding dynamic filters
Lecture 52 Creating dashboard intro
Lecture 53 Warehouse utilization dashboard.
Lecture 54 Analysis by warehouse
Lecture 55 Analysis for under utilization
Lecture 56 Analysis for over utilization
Lecture 57 Building dashboard
Lecture 58 Leveraging snowsight
Lecture 59 Conclusion
Section 11: /*******Module-2 starts*******/
Lecture 60 Module-2 Introduction.
Lecture 61 [Optional] Virtual machine setup
Lecture 62 [Optional] Mount folder
Lecture 63 [Optional] Connect from vs code
Lecture 64 Install AWS CLI in windows.
Lecture 65 Install AWS CLI in Mac.
Lecture 66 Install Snow CLI
Lecture 67 Things to be taken care.
Section 12: Loading data in snowflake.
Lecture 68 Introduction to stages in snowflake.
Lecture 69 Configuring connection to stage.
Lecture 70 Types of internal stage
Lecture 71 Introduction to stage object , file format and copy command.
Section 13: Load data using internal stage.
Lecture 72 Section intro.
Lecture 73 Upload data using webconsole.
Lecture 74 Scenario
Lecture 75 [Demo] Load data from local system to snowflake-Part1
Lecture 76 [Demo] Load data from local system to snowflake-Part2
Lecture 77 [Demo] Load data from local system to snowflake-Part3
Lecture 78 [Demo] Download data from snowflake to local system.
Lecture 79 [Demo] Load data using named stage object.
Section 14: Load data from AWS s3
Lecture 80 Section introduction.
Lecture 81 Create aws account.
Lecture 82 Scenario.
Lecture 83 Create aws s3 bucket.
Lecture 84 Create aws s3 policy.
Lecture 85 Create aws role.
Lecture 86 Create snowflake integration object.
Lecture 87 Query aws s3 from snowflake.
Lecture 88 Load data from s3 to snowflake.
Lecture 89 Unload data to s3
Lecture 90 Aws cli basics
Lecture 91 Other options to copy.
Lecture 92 Copy zip files
Lecture 93 Upload data to aws s3 using aws web console and cli.
Lecture 94 Section Recap.
Section 15: Snowflake external tables
Lecture 95 Introduction to external tables.
Lecture 96 Why external tables.
Lecture 97 Insert scenario.
Lecture 98 Delete scenario.
Lecture 99 Update scenario.
Lecture 100 Partition in external tables.
Lecture 101 Manual partition in external tables.
Lecture 102 Auto refresh metadata.
Section 16: Load data from Google GCS.
Lecture 103 Create gcs account.
Lecture 104 Create bucket.
Lecture 105 Copy command anatomy.
Lecture 106 Create Gcp storage integration object.
Lecture 107 Query Gcs data.
Lecture 108 Load data into snowflake.
Lecture 109 Unload Gcs data.
Section 17: Loading data from Azure.
Lecture 110 Create azure account
Lecture 111 Connecting snowflake to azure blob storage
Lecture 112 Upload data using AZCOPY
Lecture 113 Query parquet data from azure
Lecture 114 Load parquet data to snowflake
Lecture 115 Unload parquet data to azure
Section 18: Copy options
Lecture 116 Introduction.
Lecture 117 [Demo] Validate before loading data to table.
Lecture 118 [Demo] File level options.
Lecture 119 [Demo] Collecting rejected records from copy command.
Lecture 120 [Demo] Enforce and Truncate column option.
Lecture 121 [Demo] Force option.
Lecture 122 [Demo] Purge option.
Lecture 123 Load history and copy history views
Section 19: [Demo] Working with copy command.
Lecture 124 Preparing data.
Lecture 125 Load sample data.
Lecture 126 Copy command performance.
Section 20: Assignments.
Lecture 127 Assignment document: To load data to two different tables.
Lecture 128 Assignment document : Unload data to internal stage.
Lecture 129 Assignment document : Create integration object.
Lecture 130 Assignment document : Using snowflake as query engine.
Lecture 131 Assignment document : Loading aws open data.
Section 21: Loading unstructured data
Lecture 132 Loading unstructured data introduction.
Lecture 133 Load json data
Lecture 134 Load xml data
Lecture 135 Loading unstructured data: Method-1
Lecture 136 Loading unstructured data : Method-2
Lecture 137 Loading unstructured data : Method-3
Section 22: /*******Module-3 starts*******/
Lecture 138 Module 3 introduction.
Section 23: Snow pipe in snowflake.
Lecture 139 Snow pipe introduction.
Lecture 140 Snowpipe demo
Lecture 141 Snowpipe analysis part-1
Lecture 142 Snowpipe analysis part-2
Lecture 143 Snowpipe analysis part-3
Section 24: Data sharing in snowflake
Lecture 144 Data sharing quick introduction.
Lecture 145 Data share demo
Lecture 146 Creating secure view
Lecture 147 Difference between secure and normal view.
Lecture 148 Creating snowflake reader account.
Lecture 149 Configure reader account.
Lecture 150 Restriction on reader account.
Lecture 151 Data share significance.
Lecture 152 Further reading Resources
Section 25: Time Travel in snowflake
Lecture 153 Time travel demo.
Lecture 154 Time travel in snowflake and how it works.
Lecture 155 Table retention period.
Lecture 156 More on time travel.
Section 26: Fail safe in snowflake
Lecture 157 Fail safe in snowflake.
Lecture 158 Access fail safe from web console.
Lecture 159 Relation between fail safe and retention period.
Lecture 160 Table types in snowflake.
Lecture 161 Permanent table and failsafe.
Lecture 162 Transient table.
Lecture 163 Temporary table.
Lecture 164 Creating transient database in snowflake.
Lecture 165 Fail safe further reading.
Section 27: Clone feature in snowflake
Lecture 166 Clone feature in snowflake and how it works.
Lecture 167 Clone feature demo.
Lecture 168 Table swap in snowflake.
Section 28: Data sampling in snowflake.
Lecture 169 Snowflake sampling introduction.
Lecture 170 Row vs Block sampling.
Lecture 171 Sampling Demo
Lecture 172 Clone vs Sampling.
Lecture 173 Reference- Sampling
Section 29: Scheduling in snowflake-TASKS
Lecture 174 Snowflake tasks introduction.
Lecture 175 Snowflake task demo.
Lecture 176 Task dependency intro.
Lecture 177 Task dependency demo
Section 30: Streaming in snowflake.
Lecture 178 Introduction to streams.
Lecture 179 Streaming demo part 1
Lecture 180 Streaming demo part 2
Lecture 181 Types of stream
Lecture 182 CHANGE clause
Lecture 183 Slowly changing dimension using streams.
Section 31: [Demo] continuous data load in snowflake
Lecture 184 Demo overview
Lecture 185 Code preparation
Lecture 186 Execution
Section 32: Materialized views
Lecture 187 Materialized view intro
Lecture 188 MV under the hood
Lecture 189 MV maintenance cost
Lecture 190 Before using MV
Lecture 191 Limitation of Materialized views.
Section 33: /*******Module-3 ends*******/
Lecture 192 Module-3 ends.
Section 34: /****Module-4****/
Lecture 193 Data governacence and Administration
Section 35: Data masking
Lecture 194 Introduction to data masking.
Lecture 195 Understand data masking
Lecture 196 Data masking scenario 1
Lecture 197 Data masking scenario 2
Lecture 198 Segregation of duties
Lecture 199 Apply data masking (HIPAA version 1)
Lecture 200 HIPAA rules version 2
Lecture 201 Problem with masking
Lecture 202 Using views
Lecture 203 Using views ( continued )
Section 36: Row access policy
Lecture 204 Introduction.
Lecture 205 Row access policy demo
Lecture 206 why row access policy?
Lecture 207 Segregation of duties
Lecture 208 Mapping tables.
Lecture 209 Dml operations
Lecture 210 Conclusion.
Section 37: Tagging in snowflake
Lecture 211 Tagging in snowflake
Lecture 212 Create and apply tag
Lecture 213 Seggregation of duties
Lecture 214 Prepare environment
Lecture 215 Applying tag
Lecture 216 Tag based masking policy part1
Lecture 217 Tag based masking policy part2
Lecture 218 Masking based on tag value.
Lecture 219 Classification
Lecture 220 Classification demo
Section 38: Stored procedures
Lecture 221 Section introduction
Lecture 222 Stored procedure basics
Lecture 223 Returning result from stored procedure
Lecture 224 Using control statements.
Lecture 225 Using bind variables.
Lecture 226 Table fill rate procedure.
Lecture 227 Error scenarios.
Lecture 228 Handling error scenarios.
Lecture 229 Logging error messages.
Lecture 230 Transactions in stored procedures.
Lecture 231 Caller and owner procedures.
Lecture 232 Using session variables in procedures.
Lecture 233 Owner and Caller procedure use cases.
Section 39: Access control in snowflake
Lecture 234 Access-control introduction.
Lecture 235 Objects and role hierarchy.
Lecture 236 Default roles.
Lecture 237 Creating database: Level1
Lecture 238 Creating database : Level2
Lecture 239 Creating database : Level3
Lecture 240 Creating database : Level4
Lecture 241 Creating database : Level5
Lecture 242 Managed schema introduction.
Lecture 243 Managed schema demo.
Lecture 244 Building hierarchy introduction.
Lecture 245 Building hierarchy demo.
Lecture 246 Hierarchy for analyst.
Lecture 247 Future grants.
Lecture 248 SharedDB objects
Lecture 249 Summary.
Section 40: Snowflake best practices
Lecture 250 Best practices introduction.
Lecture 251 Best practices while doing clone.
Lecture 252 Best practice while creating database and tables.
Lecture 253 Best practice while using virtual warehouse.
Lecture 254 Best practice while using retention period and time travel.
Section 41: Bonus section
Lecture 255 Snowflake with spark
Developers planning to learn snowflake database.,Architects who want to understand best practices followed while using snowflake database.
Course Information:
Udemy | English | 20h 44m | 9.30 GB
Created by: Pradeep H C
You Can See More Courses in the IT & Software >> Greetings from CourseDown.com