Apache NiFi Complete Master Course HDP Automation ETL
What you’ll learn
Apache Nifi (Niagara Files) basics to advanced concepts
Flowfile, Processor, Connections, Controller, ProcessGroup, Input – output ports, Funnel etc.,
Installation, Security, Customization, Scalability of Apache Nifi
Develop simple to complex Dataflow and take it to production
Nifi Registry – Dataflow registry
Hortonworks DataFlow HDF
Integreate with Kafka, NoSQL Database, RDBMS, File System, etc
Porcess different types of files like CSV, JSON, Text file, etc.,
Requirements
Basic understanding on Data movement and ETL
Interest to learn more and upgrade to latest technology
Description
Apache Nifi is next generation framework to create data pipeline and integrate with almost all popular systems in the enterprise. It has more than 250 processors and more than 70 controllers. This course covers all all basic to advanced concepts available in Apache Nifi likeFlowfileControllersProcessorsConnectionsProcess GroupFunnelData ProvenanceProcessor relationshipsInput and Output PortsThis course also covers on the Apache Nifi Subprojects like Nifi RegistryAs part of production maintenance, user may have to take cautious decision to improve the performance and handle the errors efficiently. To facilitate the same, Demo also covers on Handling Throughput and LatencyHandling Back Pressure and YieldError handlingFailure RetryMonitoring BulletinData ProvenanceTo have seamless experience with data, handling of data latency and throughput and prioritizing the data is important. Its controlled with relationship, yield and back pressure.Various processors and controllers to process various type of data is demonstrated.Processors which are used in production scenarios like HTTP, RDBMS, NoSQL S3, CSV, JSON, Hive, etc., are covered in detail along with controllers like SSL, ConnectionPool, etc., with demo.All these concepts are covered with demo and real time implementation is provided.For easy practical purpose, all the demonstrated flowfile template is uploaded as part of the course.Demo on creating and using KeyStore, Trust Store for SSL communication.Using Maven and Eclipse EE for custom processor and deploying nar file to Nifi libraries.
Overview
Section 1: Introduction to Apache Nifi
Lecture 1 Introduction
Lecture 2 Apache Nifi Introduction
Lecture 3 Dataflow Introduction – Key Features
Lecture 4 Basic Installation
Lecture 5 Terminology Introduction
Lecture 6 UI Introduction – Play with Apache Nifi User Interface
Section 2: First Baby Step – Flow file Demo
Lecture 7 Create Simple Workflow
Section 3: Processors and Connections
Lecture 8 Processor Category
Lecture 9 Connection configuration
Lecture 10 Processor Configuration Settings
Lecture 11 Processor Configuration Scheduling
Lecture 12 Processor Configuration Property
Section 4: Next Step into Flowfile
Lecture 13 Working with Attributes
Lecture 14 Log Configuration and Monitoring Logs
Lecture 15 Handling Failures
Lecture 16 Working With Templates
Section 5: Integrating Apache Nifi with Distributed Messaging System – Apache Kafka
Lecture 17 Apache Kafka Quick Introduction and Demo
Lecture 18 Nifi As Producer
Lecture 19 Nifi As Consumer
Section 6: Process group and Funnel
Lecture 20 Process group – Input and Output ports
Lecture 21 Funnel Forking
Lecture 22 Funnel Combine
Section 7: Monitoring and Provenance
Lecture 23 Nifi Monitoring and Statistics
Lecture 24 Data Provenance
Section 8: Structured Data Processing
Lecture 25 Read MySQL Table data as Avro and JSON
Lecture 26 Transform CSV to JSON
Lecture 27 Managing state with MySQL and Incremental Fetch
Lecture 28 Transform CSV to JSON using dynamic schema
Section 9: Nifi Registry
Lecture 29 Apache Nifi Registry – Introduction
Lecture 30 Nifi Registry as Version Control System
Section 10: Nifi Cluster
Lecture 31 Cluster Installation and Configuration
Lecture 32 Cluster Flow File Demo
Section 11: Nifi and Bigdata Ecosysem
Lecture 33 Nifi HDFS Interaction
Lecture 34 Nifi Hive Interaction
Section 12: HTTP Processors
Lecture 35 HTTP Processor Introduction
Lecture 36 GetHTTP Processor
Lecture 37 PostHTTP Processor and SSL Context Service setup
Lecture 38 ListenHTTP Processor
Lecture 39 InvokeHTTP Processor without SSL
Lecture 40 InvokeHTTP with SSL
Section 13: Nifi and AWS
Lecture 41 AWS S3 add Object with PutS3Object Processor
Lecture 42 AWS S3 list objects with ListS3 Processor
Lecture 43 AWS S3 add object – using AWS Controller service
Section 14: Nifi and NoSQL Database
Lecture 44 Mongo DB put records with Nifi
Section 15: Nifi and Apache Solr
Lecture 45 Apache Solr Introduction, Installation and Configuration
Lecture 46 Apache Solr Content Stream Demo
Section 16: Custom Processor and Custom Controller
Lecture 47 Project setup with Maven and Eclipse
Lecture 48 Build project and generate nar file
Lecture 49 Create sample workflow with custom processor and validate
Lecture 50 Create Custom Controller
Lecture 51 Use custom controller within custom processor
Section 17: Practical Use Cases
Lecture 52 Use case 1 : Extract data from fordgobike, transform and store it in HDFS as CSV
Lecture 53 Use Case 2 : Part 1 : Extract Twitter data to Apache Solr
Lecture 54 Uase Case 2 : Part 2 : Visualize Twitter data using Banana Dashboard
Section 18: Reference Resources
Lecture 55 Test Data used
Lecture 56 Course Slides
Section 19: Bonus Lecture
Lecture 57 Special coupon to join my other courses
Developers, Architects, Beginners who wants to learn Apache NiFi,ETL team who wants to move to latest technology
Course Information:
Udemy | English | 5h 3m | 2.73 GB
Created by: MUTHUKUMAR Subramanian
You Can See More Courses in the IT & Software >> Greetings from CourseDown.com