Web Scraping in Python With BeautifulSoup and Selenium 2022

Web Scraping in Python With BeautifulSoup and Selenium 2022

What you’ll learn

Understanding the fundamentals of Web Scraping
Build your own web scraping projects
Learn core components of two of the most powerful scraping libraries: BeautifulSoup and Selenium
How to click on a button, send text to an input box, and self-scroll using Selenium
Scraping data off of single page, multiple page, and infinite scrolling websites
5 projects each with it’s own unique challenge
Automate python scripts
Understand HTML and Xpath selectors

Requirements

Beginners knowledge of Python
Internet Access

Description

Web Scraping has become one of the hottest topics in the data science world, for getting access to data can make or break you.This is why Fortune 500 companies like Walmart, CNN, Target, and Amazon use web scraping to get ahead and stay ahead with data.It’s the original growth tool and one of their best-kept secrets…And it can easily be yours too.Welcome to Web Scraping in Python with BeautiuflSoup and Selenium!The most up to date and project-oriented course out there currently.In this course, you’re going to learn how to scrape data off some of the most well-known websites which include:TwitterAirbnbNikeGoogleIndeedNFLMarketWatchWorldometersIMDbCarpagesAt the end of this course, you will understand the most important components of web scraping and be able to build your own web scrapers to obtain new data from any website, automate any task using web scraping, and more.Plus, familiarize yourself with some of the most common scraping techniques and sharpen your Python programming skills while you’re at it!First, learn the essentials of web scraping, explore the framework of a website, and get your local environment ready to take on scraping challenges with BeautifulSoup, and Selenium.Next, cover the basics of BeautifulSoup, utilize the requests library and LXML parser, and scale up to deploy a new scraping algorithm to scrape data from any table online, and from multiple pages.Third, set up Selenium to deal with JavaScript-driven webpages, and use the unique functions of Selenium to interact with pages. Combine the concepts of BeautifulSoup and Selenium to create the most effective scrapers to deal with some of the most challenging websites.Finally, learn how to make web scraping fully automatic by running your scraper at a specific time each day.What makes this course different from the others, and why you should enroll?First, this is the most updated course currently outSecond, this is the most project-based course you will find, where we will scrape many of the internets most well-known websitesYou will have an in-depth step by step guide on how to become a professional web scraper.You will learn how to use Selenium to scrape JavaScript websites and I can assure you, you won’t find any tutorials out there that teach you how to really use Selenium like I’ll be doing in this course.You will learn how to create a fully automated web scraping script that runs periodically without any intervention from you.30 days money-back guarantee by UdemySo whether you’re a data scientist, machine learning, or AI engineer who wants to access more data sources; a web developer looking to automate tasks, or a data buff with a general interest in data science and web scraping…This course delivers an in-depth presentation of web scraping basics, methodologies, and approaches that you can easily apply to your own personal projects, or out there in the real world of business.Join me now and let’s start scraping the web together. Enroll today.

Overview

Section 1: Introduction

Lecture 1 Applications Of Web Scraping

Lecture 2 Overview of Web Scraping

Lecture 3 Web Scraping Example – Twitter

Lecture 4 BONUS: 1-ON-1 Mentorship

Lecture 5 Installing Python

Lecture 6 Installing Packages

Lecture 7 Fixing Video Quality – MUST WATCH!

Lecture 8 Asking Questions

Section 2: How Websites are Displayed

Lecture 9 How Websites are Displayed

Lecture 10 Installing Google Chrome

Section 3: Basics of BeautifulSoup

Lecture 11 Section Overview

Lecture 12 How To Get The HTML

Lecture 13 Tags

Lecture 14 Navigable Strings

Lecture 15 Attributes

Lecture 16 Comments

Section 4: Searching and Extracting From the HTML

Lecture 17 Section Overview

Lecture 18 find()

Lecture 19 find_all() – Part 1

Lecture 20 find_all() – Part 2

Lecture 21 find_all() – Part 3

Lecture 22 Extracting Data From Nested HTML Tags

Lecture 23 Coding Exercise – Stocks

Lecture 24 Answer – Coding Exercise

Section 5: Project #1 – Scraping a Table

Lecture 25 Section Overview

Lecture 26 Scraping a Table – Part 1

Lecture 27 Scraping a Table – Part 2

Lecture 28 Scraping a Table – Part 3

Lecture 29 Coding Exercise – NFL Stats

Lecture 30 Answer – Coding Exercise

Section 6: Project #2 – Dealing with Multiple Pages

Lecture 31 Section Overview

Lecture 32 Dealing with Multiple Pages – Part 1

Lecture 33 Dealing with Multiple Pages – Part 2

Lecture 34 Dealing with Multiple Pages – Part 3

Lecture 35 Dealing with Multiple Pages – Part 4

Lecture 36 Coding Exercise – Carpages

Lecture 37 Answer – Coding Exercise

Lecture 38 Note: Answer – Coding Exercise

Section 7: JavaScript Driven Webpages

Lecture 39 Section Overview

Lecture 40 JavaScript Driven Webpages

Lecture 41 Selenium

Lecture 42 Installing Selenium

Section 8: Selenium

Lecture 43 Section Overview

Lecture 44 Using the Web Driver

Lecture 45 Xpath

Lecture 46 find_element()

Lecture 47 Sending Text Into an Input Box

Lecture 48 Clicking On a Button

Lecture 49 Taking a Screenshot

Lecture 50 Self-Scrolling

Lecture 51 Wait Times

Lecture 52 Coding Exercise – Imdb

Lecture 53 Answer – Coding Exercise

Lecture 54 Issue With My Downloadable Code

Section 9: Project #3 – Infinite Scrolling

Lecture 55 Section Overview

Lecture 56 Infinite Scrolling – Part1

Lecture 57 Infinite Scrolling – Part 2

Lecture 58 Infinite Scrolling – Part 3

Lecture 59 Infinite Scrolling – Part 4

Lecture 60 Coding Exercise – Union Los Angeles

Lecture 61 Answer – Coding Exercise

Section 10: Project #4 – Twitter

Lecture 62 Section Overview

Lecture 63 Twitter – Part 1

Lecture 64 Twitter – Part 2

Lecture 65 Twitter – Part 3

Lecture 66 Twitter – Part 4

Lecture 67 Twitter – Part 5

Lecture 68 Coding Exercise – Indeed

Lecture 69 Answer – Coding Exercise

Section 11: Project #5 – Automating Python Scripts

Lecture 70 Section Overview

Lecture 71 Intro To Project

Lecture 72 Sending Emails Through Python

Lecture 73 Automating Any Python Script

Lecture 74 Congratulations!!!

Section 12: Bonus Lectures

Lecture 75 **HOW TO MAKE MONEY WITH WEB SCRAPING**

Anyone interested in harnessing the power of data, web scraping, and data mining,Data Scientists who want to take their skills to the next level,Web developers looking to obtain new information or automate tasks,Anyone who wants to stop spending hours manually copying data off a website, and let a computer do it for them

Course Information:

Udemy | English | 9h 22m | 4.52 GB
Created by: Christopher Zita

You Can See More Courses in the Developer >> Greetings from CourseDown.com