How to Save a Web Page with Selenium using Java?

Selenium is widely known for automating browser interactions, but it can also be used to save web pages directly. This capability is particularly useful when you need to archive web content, save dynamic pages that change frequently, or scrape and store HTML for later analysis. In this tutorial, we’ll explore how to save a web page using Selenium and Java, focusing on creating simple and efficient of code.

This article will guide you through the process of saving a web page using Selenium with Java, providing clear examples that are easily understandable even for those new to Selenium. By the end, you'll have a working solution that you can adapt to your specific needs.

Example to Save a Web Page with Selenium using Java

Here is the following examples that can save a web page with Selenium using Java:

package com.example.tests;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import io.github.bonigarcia.wdm.WebDriverManager;

import java.io.FileWriter;
import java.io.IOException;

public class ZerodhaPageSaveTest {
    public static void main(String[] args) throws InterruptedException, IOException {
        // Set up ChromeDriver using WebDriverManager
        WebDriverManager.chromedriver().setup();

        // Initialize ChromeDriver instance
        WebDriver driver = new ChromeDriver();

        // Maximize the browser window
        driver.manage().window().maximize();

        // Navigate to Zerodha
        driver.get("https://zerodha.com/");

        // Get the page source
        String pageSource = driver.getPageSource();

        // Directory where the file will be saved
        String directoryPath = "change the path where you want to save";
        // File path
        String filePath = directoryPath + "/saved_webpage.html";

        // Save the page source to a file
        try (FileWriter fileWriter = new FileWriter(filePath)) {
            fileWriter.write(pageSource);
            System.out.println("Webpage saved successfully at: " + filePath);
        } catch (IOException e) {
            System.out.println("Error while saving webpage: " + e.getMessage());
        }

        // Close the browser
        driver.quit();
    }
}

Explanation:

1. WebDriverManager is used to set up the ChromeDriver automatically, eliminating the need to manually download and manage browser drivers.
2. driver.get("Sites which you want to save") navigates to the desired web page.
3. driver.getPageSource() retrieves the HTML content of the page.
4. The HTML content is then written to a file named saved_page.html using a FileWriter.

Output

Conclusion

Saving a web page with Selenium using Java is a simple yet powerful way to archive content, scrape data, or perform web-related tasks that require offline access to web pages. By following the steps in this tutorial, you now have a functional script that can be easily adapted to save any web page you need. Whether you’re a beginner or an experienced developer, this method provides a reliable and efficient solution for saving web pages using Selenium.