Developing a Multithreaded Kernel From Scratch

Developing a Multithreaded Kernel From Scratch

What you’ll learn

How to create a kernel from scratch
How to create a multi-tasking kernel
How to handle malicious or problematic programs in your operating system. Terminating them if they misbehave.
How memory works in computers
The difference between kernel land, user land and the protection rings that make up modern computing
Kernel design patterns used by the Linux kernel its self
You will learn all about virtual memory and how to map virtual addresses to physical addresses
You will learn how to make the kernel understand processes and tasks
You will learn how to load ELF files
You will learn how to debug disassembled machine code
You will learn how to debug your kernel in an emulator with GDB.

Requirements

You must know the C programming language
It is wise have some basic knowledge in assembly language
You should have a Linux operating system, free to install from the internet (We use Ubuntu in this course)

Description

This course is designed to teach you how to create your very own multitasking operating system and kernel from scratch. It is assumed you have no experience in programming kernels and you are taught from the ground up.Real Mode DevelopmentReal mode is a legacy mode in all Intel processors that causes the processor to start in a legacy state, it performs like the old 8086 Intel processors did back in the way.In the “Real Mode Development” section of the course we start by learning about the boot process and how memory works, we then move on to creating our very own boot loader that we test on our real machine! This boot loader will output a simple “Hello World!” message to the screen and we write this boot loader in purely assembly language.In this section we also read a sector(512 bytes) from the hard disk and learn all about interrupts in real mode and how to create them.This section gives you a nice taster into kernel development, without over whelming you with information. You are taught the basics and enough about the legacy processors to be able to move forward to more modern kernel development further into this course.Protected Mode DevelopmentIn this section we create a 32 bit multi-tasking kernel that has the FAT16 filesystem. Our kernel will use Intel’s built in memory protection and security mechanisms that allow us to instruct the processor to protect our kernel and prevent user programs from damaging it.This section is very in depth, you are taught all about paging and virtual memory. We take advantage of clever instructions in Intel processors to allow all processes to share the same memory addresses, this is known as memory virtualization. We map memory addresses to point to different physical memory addresses to create the illusion that every process that is running is loaded at the same address. This is a very common technique in kernel development and is also how swap files work (Those files that are used to compensate for when you run out of usable RAM).We create our own virtual filesystem layer that uses a design that is similar to the Linux kernel. This clever abstraction that will be taught to you was inspired by the instructors knowledge of writing Linux kernel drivers in his past. You are taught about the design of the FAT16 filesystem and how the FAT16 filesystem is broken down into clusters and that they can chain together. We then implement our very own FAT16 filesystem driver allowing files to be born!We implement functionality for tasks and processes and write our own keyboard drivers.In this course you also get to learn how memory management works, we implement the “malloc” and “free” functions creating our very own heap that’s designed to keep track of what memory is being used. Memory management is essential in any operating system and kernel.Let us not forget that we even create an ELF file loader, we will compile all our operating systems programs into ELF files and allow the loading of binary programs or ELF programs. ELF files contain a lot of information that describes our program for example where our program should be loaded into memory and the different sections of the program.By the end of this course you will have a fully functioning 32 bit multi-tasking kernel that can have many processes and tasks running at the same time. You will have a working shell that we can use as well.Assembly language bonusThis is a bonus section designed to bring your assembly skills up to scratch should you struggle a little bit with the assembly language in this course. It’s however advised you come to this course with experience in assembly language, we do use it and its important. Never the less if you want to take a chance on this course with no assembly experience then this section will help point you in the right direction so your able to take what you learned and apply it to the kernel.Taught by an expert that has created Linux kernel modules professionally in the work place. 15 Years Experience As A Software Engineer

Overview

Section 1: Introduction

Lecture 1 Introduction

Section 2: Setup And A Brief Explanation Of Kernel Development

Lecture 2 What Is Memory?

Lecture 3 The Boot Process

Lecture 4 Installing What We Need For Real Mode Development

Section 3: Real Mode Development

Lecture 5 Hello World Bootloader

Lecture 6 Understanding Real Mode

Lecture 7 Segmentation Memory Model

Lecture 8 Improving Our Bootloader

Lecture 9 Preparing our bootloader to be booted on real hardware

Lecture 10 Writing our bootloader to a USB stick

Lecture 11 Booting the bootloader

Lecture 12 The Interrupt Vector Table Explained

Lecture 13 Implementing our own interrupts in real mode

Lecture 14 Disk Access And How It Works

Lecture 15 Reading from the hard disk

Section 4: Protected Mode Development

Lecture 16 What is Protected Mode?

Lecture 17 Switching To Protected Mode

Lecture 18 Restructuring Our Project

Lecture 19 Enabling the A20 line

Lecture 20 Creating a Cross Compiler So We Can Code In C

Lecture 21 Loading our 32 bit kernel into memory and working with debugging symbols

Lecture 22 Cleaning our object files

Lecture 23 Dealing With Alignment Issues

Lecture 24 C Code In Protected Mode

Lecture 25 Text Mode Explained

Lecture 26 Writing To The Screen, Hello World Tutorial

Lecture 27 Interrupt Descriptor Table Explained

Lecture 28 Implementing The Interrupt Descriptor Table

Lecture 29 Implementing In and Out Functions

Lecture 30 Programmable Interrupt Controller Explained

Lecture 31 Programmable Interrupt Controller Implementation

Lecture 32 Understanding The Heap And Memory Allocation

Lecture 33 Implementing Our Heap

Lecture 34 Creating the enable interrupts function

Lecture 35 Understanding Paging

Lecture 36 Implementing Paging

Lecture 37 Modifying the page table

Lecture 38 Preparing To Read From The Hard Disk

Lecture 39 Reading from the disk in C with the ATA controller

Lecture 40 Improving Our Disk Driver

Lecture 41 What is a filesystem?

Lecture 42 Creating a path parser

Lecture 43 Creating a disk stream

Lecture 44 File Allocation Table Explained

Lecture 45 Starting To Create our FAT File system

Lecture 46 Understanding the VFS(Virtual File System) Layer

Lecture 47 Implementing our virtual filesystem core functionality

Lecture 48 implementing FAT16 filesystem driver core functionality

Lecture 49 Implementing FAT16 Structures

Lecture 50 Implementing The FAT16 Resolver Function

Lecture 51 Implementing the VFS fopen function

Lecture 52 Implementing FAT16 fopen function

Lecture 53 Implementing the VFS fread function

Lecture 54 Implementing FAT16 fread functionality

Lecture 55 Implementing the VFS fseek functionality

Lecture 56 Implementing the FAT16 fseek functionality

Lecture 57 Implementing the fstat VFS functionality

Lecture 58 Implementing the FAT16 fstat function

Lecture 59 Implementing the VFS fclose functionality

Lecture 60 Implementing the FAT16 fclose functionality

Lecture 61 Implementing a kernel panic

Lecture 62 Understanding User Land

Lecture 63 Changing our kernel segment and data descriptors to be written in C

Lecture 64 Implementing The TSS(Task Switch Segment)

Lecture 65 Implementing Task Foundations

Lecture 66 Implementing Process Foundations Part 1

Lecture 67 Implementing Process Foundations Part 2

Lecture 68 Packing the GDT

Lecture 69 Implementing User Land Functionality

Lecture 70 Creating our first user process application

Lecture 71 Executing the process and dropping into user land privileges

Lecture 72 Changing the paging functionality

Lecture 73 Talking with the kernel from userland

Lecture 74 Creating the interrupt 0x80 for user process to kernel communication

Lecture 75 Creating the ability to create and execute kernel commands

Lecture 76 Creating our first kernel command

Lecture 77 Calling our kernel command

Lecture 78 Copying strings from the tasks process

Lecture 79 Reading the task’s stack

Lecture 80 Creating the print command in the kernel

Lecture 81 Understanding keyboard access in protected mode

Lecture 82 Creating the virtual keyboard layer

Lecture 83 Creating the PS2 port keyboard driver part 1

Lecture 84 Improving our interrupt descriptor table design

Lecture 85 Creating a cleaner way to create interrupt handlers in the interrupt descriptor

Lecture 86 Changing The Current Process

Lecture 87 Creating the PS2 port keyboard driver part 2

Lecture 88 Getting a key from the keyboard buffer in user land

Lecture 89 Creating a putchar command that writes one character to the terminal

Lecture 90 Implementing backspace in the terminal

Lecture 91 Revising our stream reader

Lecture 92 Elf Files Explained

Lecture 93 Implementing The Elf Loader – Part 1

Lecture 94 Implementing The Elf Loader – Part 2

Lecture 95 Implementing The Elf Loader – Part 3

Lecture 96 Implementing The Elf Loader – Part 4

Lecture 97 Implementing The Elf Loader – Part 5

Lecture 98 Implementing The Elf Loader – Part 6

Lecture 99 Writing User Programs In C

Lecture 100 Implementing system print in stdlib

Lecture 101 Implementing system get key in stdlib

Lecture 102 Implementing Malloc In Our stdlib

Lecture 103 Implementing Free In Our stdlib

Lecture 104 Changing the way we map virtual pages for the process

Lecture 105 Implementing itoa function

Lecture 106 Implementing the putchar function

Lecture 107 Implementing the printf function

Lecture 108 Implementing the ability to read lines

Lecture 109 Creating a shell

Lecture 110 Loading other programs from our shell

Lecture 111 Creating some important stdlib functions

Lecture 112 Memory Mapping malloc in stdlib

Lecture 113 Memory Unmapping free In stdlib

Lecture 114 Process arguments – Part 1

Lecture 115 Process Arguments – Part 2

Lecture 116 Process Arguments – Part 3

Lecture 117 Implementing A ‘System’ Command

Lecture 118 Implementing program termination

Lecture 119 Handling program crashes

Lecture 120 Creating an exit command

Lecture 121 Handling caps lock, upper case and lower case letters

Lecture 122 Running multiple tasks at the same time multi-tasking

Section 5: Optional

Lecture 123 README

Lecture 124 Changing our fat16_new_fat_item_for_directory_item function

Lecture 125 Changing our fat16_open function

Lecture 126 Changing our fat16_get_root_directory function

Lecture 127 Changing our process_load_binary function

Lecture 128 Improvements to our fat16_to_proper_string function

Lecture 129 Changing our restore_general_purpose_registers function

Section 6: Assembly Language Catchup

Lecture 130 Assembly Catchup!

Lecture 131 What is assembly language?

Lecture 132 Installing the emulator

Lecture 133 Hello World In Assembly

Lecture 134 Transistors And Logic Gates Understanding The Processor

Lecture 135 Registers in the 8086

Lecture 136 Segmentation Memory Model Explained

Lecture 137 The Stack, Subroutines And Endiness Explained

Section 7: Conclusion

Lecture 138 Bonus Lecture

Beginner kernel developers who want to learn how to create kernels

Course Information:

Udemy | English | 28h 14m | 15.00 GB
Created by: Daniel McCarthy

You Can See More Courses in the Developer >> Greetings from CourseDown.com