The Complete Apache Kafka Practical Guide

Learn core Apache Kafka features, create Java, Node.js, Python Producers and Consumers – Full Kafka hands-on experience!
The Complete Apache Kafka Practical Guide
File Size :
5.53 GB
Total length :
8h 39m



Bogdan Stashchuk | Software Engineer, MBA, PhD


Last update




The Complete Apache Kafka Practical Guide

What you’ll learn

Understand how Apache Kafka works
Create fault-tolerant clusters with topics replication across multiple brokers
Apache Kafka Architecture
How to create Kafka cluster with multiple Brokers
Practice to create producers and consumers using built-in console producer and console consumer
Write custom producer and consumer in Java
Use Python and Node to produce and consume messages
Learn components of the Apache Kafka Cluster

The Complete Apache Kafka Practical Guide


You could have completely no prior knowledge about Apache Kafka
Prepare only your Mac, Windows or Linux/Unix computer. Anything will work.
Also you could know nothing about Java, Node or Python. I will explain all of these from beginning
We will start from the very beginning by installing Apache Kafka and exploring it’s components


This course is all about Apache KafkaUnderstand HOW Apache Kafka works and learn its core features ON PRACTICE. This is 80% practice course without no useless demos!Build custom Apache Kafka Producers and Consumers using native Java API. Also you will build projects using APIs for other programming languages like Node.js and Python.Become a master of Apache Kafka by understanding and practicing its architecture and main features.All project files and mostly used commands are available in the GitHub repository.This is the most complete practical Apache Kafka guide here on Udemy that includes tons of practical activities. Most important is that you will learn how Apache Kafka works and knowing it you will be able much more easier use its features and fix mistakes in the future after you’ll finish this course. You can have zero knowledge about Apache Kafka, Java, Node.js or Python. All will be taught from scratch, from basic to advanced features. If you want to get deep knowledge of Apache Kafka this course is for you!We will start by installing Apache Kafka on your computer, VPS (Virtual Private Server) or Virtual Machine on your computer. You will learn that installation of Apache Kafka is pretty easy – you just need to download archive with executable scripts and run them. Also you will learn and practice how to run multiple brokers on the same computer.Afterwards we will jump into tons of practice activities and use different Apache Kafka features and built-in scripts. You will launch Zookeeper, multiple Brokers, Console Consumer and Console Producer. Also you will test performance of the Kafka Cluster using built-in utility called Performance Monitor.In practice sections you will perform multiple practice Apache activities:Create cluster with multiple brokersCreate topic with multiple partitions spread across different brokersCreate topics with replication factor that allows you to store copy of every message on different brokers for redundancyProduce messages using built-in Console ProducerConsume messages using built-in Console ConsumerLaunch multiple consumers in the same consumer groupLaunch Performance Monitor for testing Consumers and Producers performance and speedYou will also learn and practice how to use Apache Kafka API to create your own Consumers and ProducersCreate Java Maven projectLaunch Producer and Consumer using JavaLaunch multiple consumers in the same Consumer GroupUnderstand and practice difference between “subscribe” and “assign”Create Node.js projectLaunch Producers and Consumers using Node.jsCreate Python projectLaunch Producers and Consumers using PythonDuring the course you will need to view and edit text files. For that you will NOT use terminal. Instead you will use GUI application VisualStudio Code. Also all configuration files and project files are available in the GitHub repository. This means that during this course you will also learn how to use :Git and GitHubVisualStudio CodeWith this course you will get lifetime-long access to more than 100 lectures and tens of practical exercises. After the course you will become a guru of Apache Kafka and will be able easily create custom Apache Kafka Producers and Consumers. But most important is that you will UNDERSTAND Apache Kafka.You will also get 30-days money-back guarantee. No questions asked!Don’t wait and join the course now!


Section 1: Introduction

Lecture 1 Let’s get connected! Join the Learning Community

Section 2: Apache Kafka Installation Overview

Lecture 2 Apache Kafka Installation Overview

Lecture 3 Installing Apache Kafka on the Mac and Unix-like systems

Lecture 4 Installing Apache Kafka on the Mac

Lecture 5 Installing Ubuntu on MacOS using VirtualBox

Section 3: Installing Apache Kafka on the remote Ubuntu server

Lecture 6 SECTION 2 Introduction

Lecture 7 Creating remote Ubuntu Virtual Private Server

Lecture 8 Installing Apache Kafka on Virtual Private Server

Section 4: Installing Apache Kafka on Windows

Lecture 9 SECTION 3 Introduction

Lecture 10 Installing Apache Kafka on Windows

Lecture 11 Starting Zookeeper and Kafka server on Windows

Lecture 12 Installing Ubuntu on Windows using VirtualBox

Lecture 13 Installing Apache Kafka on Ubuntu using GUI

Section 5: Starting Apache Zookeeper and Kafka Broker

Lecture 14 SECTION 4 Introduction

Lecture 15 Observing contents of the Kafka folder

Lecture 16 Reading and editing Kafka files using VisualStudio Code

Lecture 17 Trying to start Kafka Server

Lecture 18 Observing Kafka Server logs

Lecture 19 Starting Zookeeper

Lecture 20 Starting Kafka Server while Zookeeper is up and running

Lecture 21 Observing logs folder and current kafka server setup

Section 6: Creating and exploring Kafka Topic

Lecture 22 SECTION 5 Introduction

Lecture 23 How to connect to Kafka cluster

Lecture 24 Create new Kafka topic

Lecture 25 What happened after creation of the new topic

Lecture 26 Read details about topic

Section 7: Producing and consuming Messages

Lecture 27 SECTION 6 Introduction

Lecture 28 Send some messages using Kafka Console Producer

Lecture 29 Consuming messages using Kafka Console Consumer

Lecture 30 Consuming messages from the beginning

Lecture 31 Running multiple consumers

Lecture 32 Running multiple producers

Lecture 33 What was changed in the Kafka logs

Section 8: What is Apache Kafka and how it works

Lecture 34 SECTION 7 Introduction

Lecture 35 What is Apache Kafka

Lecture 36 Broker

Lecture 37 Broker cluster

Lecture 38 Zookeeper

Lecture 39 Zookeeper ensemble

Lecture 40 Multiple Kafka clusters

Lecture 41 Default ports of Zookeeper and Broker

Lecture 42 Kafka Topic

Lecture 43 Message structure

Lecture 44 Topics and Partitions

Lecture 45 Spreading messages across partitions

Lecture 46 Partition Leader and Followers

Lecture 47 Controller and it’s responsibilities

Lecture 48 How Producers write messages to the topic

Lecture 49 How Consumers read messages from the topic

Section 9: GitHub Repository and Diagrams for the course

Lecture 50 SECTION 8 Introduction

Lecture 51 GitHub repository and list of basic Kafka commands

Lecture 52 Diagrams for the course

Section 10: EXAMPLE 1 – Topic with Multiple Partitions

Lecture 53 SECTION 9 Introduction

Lecture 54 If you use remote Kafka brokers

Lecture 55 Cleaning up existing kafka installation

Lecture 56 Creating topic with multiple partitions

Lecture 57 How messages were spread across different partitions

Lecture 58 Reading messages from specific partition

Lecture 59 Reading messages from specific offset in specific partition

Lecture 60 Reading details about topic and __consumer_offsets topic

Lecture 61 Summary for multiple partitions example

Section 11: EXAMPLE 2 – Kafka Cluster with Multiple Brokers

Lecture 62 SECTION 10 Introduction

Lecture 63 Example overview – run multiple brokers

Lecture 64 Creating separate configuration files for brokers

Lecture 65 Launching three brokers

Lecture 66 Getting cluster information and broker details from Zookeeper

Lecture 67 Creating multiple-partition topic in the Kafka cluster

Lecture 68 Looking at logs folders of every broker

Lecture 69 Producing and consuming messages in the cluster

Lecture 70 Details about topic in the cluster

Lecture 71 Simulating broker failure in the cluster

Lecture 72 Summary for broker cluster and topic without replication

Section 12: EXAMPLE 3 – Multiple Brokers and Topic with Replication

Lecture 73 SECTION 11 Introduction

Lecture 74 Preparing for the next example with replication

Lecture 75 Launching brokers and creating topic with replication

Lecture 76 Observing logs folder and details of the topic

Lecture 77 Producing and consuming messages in the topic with replication

Lecture 78 Observing how messages were stored in the partitions on different brokers

Lecture 79 Bringing down one of three brokers and observing changes

Lecture 80 Bringing down another broker in the cluster

Lecture 81 Bringing back both brokers

Lecture 82 Summary for replication

Section 13: EXAMPLE 4 – Kafka Consumer Groups

Lecture 83 SECTION 12 Introduction

Lecture 84 Example with consumer groups overview

Lecture 85 Exploring default consumer groups

Lecture 86 Starting consumer in the custom consumer group

Lecture 87 Starting second consumer in the same consumer group

Lecture 88 Launching one more consumer in the same group

Lecture 89 Idle consumers in the group

Lecture 90 Summary for consumer groups

Section 14: EXAMPLE 5 – Performance Testing

Lecture 91 SECTION 13 Introduction

Lecture 92 Overview of the performance testing example

Lecture 93 Starting cluster and launching basic performance test

Lecture 94 Increasing performance test parameters

Lecture 95 Testing consumer performance

Lecture 96 Getting non-zero LAG values for consumers

Lecture 97 Performance test example summary

Section 15: PROJECT 1 – Java

Lecture 98 SECTION 14 Introduction

Lecture 99 If you use remote Kafka brokers

Lecture 100 Project Files for the Java section

Lecture 101 Installing IntelliJ editor

Lecture 102 Creating and configuring Maven project

Lecture 103 Starting Kafka Cluster

Lecture 104 Creating Java Producer

Lecture 105 Continue Creating Java Producer

Lecture 106 Launching Java Producer

Lecture 107 Explaining most common Producer parameters

Lecture 108 Modifying Serializer type

Lecture 109 Producing meaningful messages with delay

Lecture 110 Refactoring Producer by adding previous example

Lecture 111 Creating consumer with autocommitting – PART 1

Lecture 112 Creating consumer with autocommitting – PART 2

Lecture 113 Consumer parameters overview

Lecture 114 Consumer with Manual Committing

Lecture 115 Consumer with Partitions Assignment

Lecture 116 Launching multipile consumers in the same consumer group

Lecture 117 CHALLENGE – Subscribe vs Assign with consumer groups

Section 16: PROJECT 2 – Node.js

Lecture 118 SECTION 15 Introduction

Lecture 119 If you use remote Kafka brokers

Lecture 120 Installing Node.js with NPM

Lecture 121 Starting up Kafka cluster with 3 brokers

Lecture 122 Initializing Node.js project

Lecture 123 Final Node.js project files

Lecture 124 Creating basic Node.js producer

Lecture 125 Producing random animal names

Lecture 126 Creating Node.js consumer

Section 17: PROJECT 3 – Python

Lecture 127 SECTION 16 Introduction

Lecture 128 If you use remote Kafka brokers

Lecture 129 Installing Python

Lecture 130 Final Python project files

Lecture 131 Launching basic Python producer

Lecture 132 Launching consumer and receiving messages

Lecture 133 Generating fake names in the messages by producer

Section 18: Course Summary

Lecture 134 Course Summary

Lecture 135 Bonus Lecture

Beginners who want to learn Apache Kafka,Advanced Apache Kafka users who want to learn how to build producers and consumers in other languages like Node or Python,This course is for you if you want to PRACTICE using Kafka and it’s APIs

Course Information:

Udemy | English | 8h 39m | 5.53 GB
Created by: Bogdan Stashchuk | Software Engineer, MBA, PhD

You Can See More Courses in the Developer >> Greetings from

New Courses

Scroll to Top