From 0 to 1 The Oozie Orchestration Framework
What you’ll learn
Install and set up Oozie
Configure Workflows to run jobs on Hadoop
Configure time-triggered and data-triggered Workflows
Configure data pipelines using Bundles
Requirements
Students should have basic knowledge of the Hadoop eco-system and should be able to run MapReduce jobs on Hadoop
Description
Prerequisites: Working with Oozie requires some basic knowledge of the Hadoop eco-system and running MapReduce jobs
Taught by a team which includes 2 Stanford-educated, ex-Googlers and 2 ex-Flipkart Lead Analysts. This team has decades of practical experience in working with large-scale data processing jobs.
Oozie is like the formidable, yet super-efficient admin assistant who can get things done for you, if you know how to ask
Let’s parse that
formidable, yet super-efficient: Oozie is formidable because it is entirely written in XML, which is hard to debug when things go wrong. However, once you’ve figured out how to work with it, it’s like magic. Complex dependencies, managing a multitude of jobs at different time schedules, managing entire data pipelines are all made easy with Oozie
get things done for you: Oozie allows you to manage Hadoop jobs as well as Java programs, scripts and any other executable with the same basic set up. It manages your dependencies cleanly and logically.
if you know how to ask: Knowing the right configurations parameters which gets the job done, that is the key to mastering Oozie
What’s Covered:
Workflow Management: Workflow specifications, Action nodes, Control nodes, Global configuration, real examples with MapReduce and Shell actions which you can run and tweak
Time-based and data-based triggers for Workflows: Coordinator specification, Mimicing simple cron jobs, specifying time and data availability triggers for Workflows, dealing with backlog, running time-triggered and data-triggered coordinator actions
Data Pipelines using Bundles: Bundle specification, the kick-off time for bundles, running a bundle on Oozie
Overview
Section 1: Introduction
Lecture 1 You, This Course and Us
Section 2: A Brief Overview Of Oozie
Lecture 2 What is Oozie?
Lecture 3 Oozie architectural components
Section 3: Oozie Install And Set Up
Lecture 4 Installing Oozie on your machine
Section 4: Workflows: A Directed Acyclic Graph Of Tasks
Lecture 5 Running MapReduce on the command line
Lecture 6 The lifecycle of a Workflow
Lecture 7 Running our first Oozie Workflow MapReduce application
Lecture 8 The job.properties file
Lecture 9 The workflow.xml file
Lecture 10 A Shell action Workflow
Lecture 11 Control nodes, Action nodes and Global configurations within Workflows
Section 5: Coordinators: Managing Workflows
Lecture 12 Running our first Coordinator application
Lecture 13 A time-triggered Coordinator definition
Lecture 14 Coordinator control mechanisms
Lecture 15 Data availability triggers
Lecture 16 Running a Coordinator which waits for input data
Lecture 17 Coordinator configuration to use data triggers
Section 6: Bundles: A Collection Of Coordinators For Data Pipelines
Lecture 18 Bundles and why we need them
Lecture 19 The Bundle kick-off time
Section 7: Installing Hadoop in a Local Environment
Lecture 20 Hadoop Install Modes
Lecture 21 Hadoop Install Step 1 : Standalone Mode
Lecture 22 Hadoop Install Step 2 : Pseudo-Distributed Mode
Section 8: Appendix
Lecture 23 [For Linux/Mac OS Shell Newbies] Path and other Environment Variables
Lecture 24 Setting up a Virtual Linux Instance – For Windows Users
Yep! Engineers, analysts and sysadmins who are interested in big data processing on Hadoop,Nope! Beginners who have no knowledge of the Hadoop eco-system
Course Information:
Udemy | English | 4h 1m | 2.00 GB
Created by: Loony Corn
You Can See More Courses in the Developer >> Greetings from CourseDown.com