Requesting a URL from a local File in Python
Last Updated :
26 Apr, 2025
Making requests over the internet is a common operation performed by most automated web applications. Whether a web scraper or a visitor tracker, such operations are performed by any program that makes requests over the internet. In this article, you will learn how to request a URL from a local File using Python.
Requesting a URL from a local File in the Python
webbrowser library would be used to include URL displaying capabilities to the python interpreter. The library could be installed using the following command:
py -m pip install webbrowser
The following file containing URLs on each line will be used for demonstration.
a file containing 3 URLs on separate linesExample 1: Requesting a URL from a local File using webbrowser
The aforementioned file would be read, and all its contents would be stored (delimited by a newline character) as members of a list where each line would contain a URL. Next, a loop would run to go over each of these URLs. In each iteration, a request to the resource located at the particular URL would be made using the webbrowser.open function. This opens the default web browser of the Operating System to display the URL.
Python3
import webbrowser
# Opening the file containing URLs in each line
# Extracting the URL from each line and storing it in a list
url_list = open('test.txt').readlines()
# This loop iterates over all the URLs present in the file
for url in url_list:
# Opening the webbrowser of the OS and displaying the URL
webbrowser.open(url)
Output:
Firstly the file containing the URLs is opened (in read mode), and all the lines in the file are stored as separate elements of a list. A loop is run on the list, which iterates over all its elements, where the elements are URLs. In each iteration, a request to the specific URL is made, and the response is displayed using the default web browser of the operating system.
The file containing the URLs needs to contain a URL on each line. Otherwise, the code must be modified to reflect such changes in the input file.
Method 2: Requesting a URL from a local File using requests
Another way of accomplishing the task at hand, without the overhead of running a browser, is by requesting the webpage and storing the response. This allows several operations that could be performed on the data obtained from the URL while not requiring the display of its contents. In the following example, the use of request module would be made to produce the desired effect.
Python3
import requests
# Opening the file containing URLs in each line
# Extracting the URL from each line and storing it in a list
url_list = open('test.txt').readlines()
# This loop iterates over all the URLs present in the file
for url in url_list:
# Making the request for the URL and storing its response
resp = requests.get(url)
# Displaying the response code & the content of the response
print(resp)
print(resp.content)
Output:
The process of extracting URLs from the file is the same as in the previous example. The only difference lies in the loop body. Inside the loop, the get function makes a request to the URL, and the response is stored in a variable. This variable could be used to obtain a lot of information regarding the URL. Firstly, the response code is displayed, which in our case is 200 (success status response code). After which the contents of the webpage received in response are displayed. If this content is stored within a file with an appropriate extension, it will act as an offline copy of the URL.
Similar Reads
Check for URL in a String - Python
We are given a string that may contain one or more URLs and our task is to extract them efficiently. This is useful for web scraping, text processing, and data validation. For example:Input:s = "My Profile: https://auth.geeksforgeeks.org/user/Prajjwal%20/articles in the portal of https://www.geeksfo
3 min read
Read File As String in Python
Python provides several ways to read the contents of a file as a string, allowing developers to handle text data with ease. In this article, we will explore four different approaches to achieve this task. Each approach has its advantages and uses cases, so let's delve into them one by one. Read File
3 min read
How to Read from a File in Python
Reading from a file in Python means accessing and retrieving the contents of a file, whether it be text, binary data or a specific data format like CSV or JSON. Python provides built-in functions and methods for reading a file in python efficiently.Example File: geeks.txtHello World Hello GeeksforGe
5 min read
User Agent in Python Request
The User-Agent (UA) string is one of the most important factors in website identification in Web Scraping. It is like a fingerprint that a client or targeted website can identify. With the help of this, you can retrieve the data by shuffling the user agent into Python requests. It typically includes
5 min read
How To Download Large File In Python With Requests
Downloading large files in Python can sometimes be a tricky task, especially when efficiency and reliability are crucial. Python's requests library provides a convenient way to handle HTTP requests, including downloading files. In this article, we will explore how to download large files in Python w
2 min read
Python | Build a REST API using Flask
Prerequisite: Introduction to Rest API REST stands for REpresentational State Transfer and is an architectural style used in modern web development. It defines a set or rules/constraints for a web application to send and receive data. In this article, we will build a REST API in Python using the Fla
3 min read
How to read specific lines from a File in Python?
Text files are composed of plain text content. Text files are also known as flat files or plain files. Python provides easy support to read and access the content within the file. Text files are first opened and then the content is accessed from it in the order of lines. By default, the line numbers
3 min read
How to Urlencode a Querystring in Python?
URL encoding a query string consists of converting characters into a format that can be safely transmitted over the internet. This process replaces special characters with a '%' followed by their hexadecimal equivalent. In this article, we will explore three different approaches to urlencode a query
2 min read
Matplotlib.axis.Axis.set_url() function in Python
Matplotlib is a library in Python and it is numerical â mathematical extension for NumPy library. It is an amazing visualization library in Python for 2D plots of arrays and used for working with the broader SciPy stack. Matplotlib.axis.Axis.set_url() Function The Axis.set_url() function in axis mod
2 min read
How to parse local HTML file in Python?
Prerequisites: Beautifulsoup Parsing means dividing a file or input into pieces of information/data that can be stored for our personal use in the future. Sometimes, we need data from an existing file stored on our computers, parsing technique can be used in such cases. The parsing includes multiple
5 min read