Python | Automate Google Search using Selenium

How to Scrape Web Data from Google using Python?

Last Updated : 26 Mar, 2024

Prerequisites: Python Requests, Implementing Web Scraping in Python with BeautifulSoup Web scraping is a technique to fetch data from websites. While surfing on the web, many websites don’t allow the user to save data for personal use. One way is to manually copy-paste the data, which both tedious and time-consuming. Web Scraping is the automation of the data extraction process from websites. In this article, we will scrape the weather update from google's search result. Modules Required

BeautifulSoup: This module is used for iterating, searching, and modifying the parse tree over the HTML or XML parser. To download it type the below command in the terminal.

pip install beautifulsoup4
       OR  
pip install bs4

Requests: Requests library is one of the integral part of Python for making HTTP requests to a specified URL. To download it type the below command in the terminal.

pip install requests

Below is the implementation.

Python3

import requests
from bs4 import BeautifulSoup

# Enter the City Name
city = input("Enter the City Name: ")
search = f"Weather in {city}"

# URL
url = f"https://www.google.com/search?q={search}"

# Sending HTTP request
req = requests.get(url)

# Pulling HTTP data from internet
sor = BeautifulSoup(req.text, "html.parser")

# Finding temperature in Celsius
temp = sor.find("div", class_='BNeawe').text

print(f'Temperature in {city} is {temp}')

Output : python-weather-data-web-scraping

Python | Automate Google Search using Selenium

akash_kumar_singh

Improve

Article Tags :

Practice Tags :

python

Similar Reads

Python | Automate Google Search using Selenium

Google search can be automated using Python script in just 2 minutes. This can be done using selenium (a browser automation tool). Selenium is a portable framework for testing web applications. It can automatically perform the same interactions that any you need to perform manually and this is a sma

Amazon product price tracker using Python

As we know Python is a multi-purpose language and widely used for scripting. We can write Python scripts to automate daily life task. Let's say we want to track any Amazon product price and grab the deal when the price changes below our defined price. Let's write a Python script for this. Below is t

Python program to Recursively scrape all the URLs of the website

In this tutorial we will see how to we can recursively scrape all the URLs from the website Recursion in computer science is a method of solving a problem where the solution depends on solutions to smaller instances of the same problem. Such problems can generally be solved by iteration, but this ne

Implementing Web Scraping in Python with Scrapy

Nowadays data is everything and if someone wants to get data from webpages then one way to use an API or implement Web Scraping techniques. In Python, Web scraping can be done easily by using scraping tools like BeautifulSoup. But what if the user is concerned about performance of scraper or need to

Python Get All Values from Nested Dictionary

In this article, we will learn how we can Get all Values from Nested Dictionary in Python Programming. A nested Dictionary in Python is a dictionary that contains another dictionary as its values. Using this, we can create a nested structure where each of the key-value pairs is in the outer dictiona

Gmail Login using Python Selenium

Python scripting is one of the most intriguing and fascinating things to do meanwhile learning Python. Automation and controlling the browser is one of them. In this particular article, we will see how to log in to the Gmail account using Python and the power of selenium. Selenium automates and cont

Python | Get a google map image of specified location using Google Static Maps API

Google Static Maps API lets embed a Google Maps image on the web page without requiring JavaScript or any dynamic page loading. The Google Static Maps API service creates the map based on URL parameters sent through a standard HTTP request and returns the map as an image one can display on the web p

Extract Data From JustDial using Selenium

Let us see how to extract data from Justdial using Selenium and Python. Justdial is a company that provides local search for different services in India over the phone, website and mobile apps. In this article we will be extracting the following data: Phone numberNameAddress We can then save the dat

Scraping Covid-19 statistics using BeautifulSoup

Coronavirus, one of the biggest pandemic has brought all the world to Danger. Along with this, it is one of the trending News, everyone has this day. In this article, we will be scraping data and printing Covid-19 statistics in human-readable form. The data will be scraped from this websitePrerequis

Python program to find GSoC organisations that use a Particular Programming Language

Currently, it's not possible to sort GSoC participating organizations by the programming languages they use in their code. This results in students spending a lot of time going through each organization's page and manually sorting through them. This article introduces a way for students to write the