Apache Kafka Spark Cassandra Real Time Streaming Project
What you’ll learn
Professionals looking for an end to end kafka, spark and cassandra streaming pipeline.
Anyone who wants to understand how to use Apache kafka in their current architecture.
Engineers who are looking to design a Kafka Solution Pipeline.
A complete case study from start to execution.
Baisc understanding of Kafka Architecure, Spark and Cassandra or SQL
The course is designed to provide a comprehensive understanding of real-time big data processing using Kafka, Spark, and Cassandra. In today’s world, data is produced at an unprecedented rate, and the ability to process and analyze this data in real-time is critical for making informed decisions. This course focuses on the fundamental concepts and architecture of Kafka, Spark, and Cassandra, and how they work together to create a robust big data processing pipeline.Students will learn how to set up Kafka clusters and work with Kafka producers and consumers. Students will also learn about Kafka Streams, a client library for building real-time streaming applications that process data directly within Kafka.Throughout the course, students will gain hands-on experience through practical exercises and projects that simulate real-world scenarios. By the end of the course, students will have a understanding of how to use Kafka, Spark, and Cassandra to build real-time big data processing systems.Course Objectives:Understand the fundamental concepts of real-time big data processingLearn the architecture setup of Kafka, Spark, and CassandraUnderstand how Kafka, Spark, and Cassandra work together to create a real-time big data processing pipelineGain hands-on experience with Kafka, Spark, and Cassandra through practical exercises and projectsLearn how to build a real-time big data processing pipeline from scratchThis course is intended for software engineers, data engineers, and data analysts who have a basic understanding of programming concepts and are familiar with SQL.
Section 1: Infrastructure Setup for the Pipeline
Lecture 1 Introduction: Project_Overview
Lecture 2 Documents
Lecture 3 Ubuntu_Machine_Creation
Lecture 4 Putty_Installation
Lecture 5 Java_Installation
Lecture 6 Kafka_Setup
Lecture 7 Hadoop_Installation
Lecture 8 SPARK SCALA Installation
Lecture 9 Cassandra_Installation
Section 2: Pipeline Setup For KAKFA SPARK Real Time Streaming
Lecture 10 JSON Data Overview and Cassandra Table Creation
Lecture 11 KAFKA Producer & Consumer Configuration
Lecture 12 Code Overview(Spark Scala)
Section 3: FINAL Pipeline Execution End to End(KAFKA + SPARK + CASSANDRA)
Lecture 13 Pipeline Execution
Lecture 14 Destroy Droplet (*Important)
Engineers looking for a CASE STUDY on Real Time Data Streaming involving KAKFA and SPARK
Udemy | English | 0h 33m | 259.48 MB
Created by: Sonal Saxena