Decoding Web Interactions: Unleashing Selenium CDP Listeners to Extract Network Responses / Blogs / Perficient

Introduction

In the dynamic realm of Selenium CDP integration, where Selenium harnesses the power of the Chrome DevTools Protocol (CDP). We previously embarked on a journey to simulate mobile browsing using Device Metrics Override: Optimizing Web Testing: Mastering Mobile Simulation with Selenium CDP Device Metrics Override / Blogs / Perficient

Building on that foundation, our exploration now takes a deeper dive into the intricate web interactions with a focus on Selenium CDP Listeners. In this article on extracting network responses with Selenium CDP integration, we will delve into harnessing network responses and request data, enabling us to extract values like status code, URL, request-id, etc.

The Crucial Role of Network Responses:

In web testing, understanding the network responses of a page is paramount. Network responses, often represented by HTTP status codes, provide insights into the success or failure of requests made by a web page. These responses, encapsulating a wealth of data beyond status codes, provide insights into resource loading, API calls, and script execution on a web page.

Selenium CDP Listeners: Unveiling Real-time Insights

Selenium, in conjunction with CDP Listeners, empowers testers to tap into this wealth of information. Listeners, as the name suggests, ‘listen’ to events within the Chrome browser and allow us to capture and analyze network responses in real-time. This functionality becomes a game-changer, providing a dynamic approach to testing and debugging.

Extracting Comprehensive Network Data with Selenium CDP

Let’s delve into a practical demonstration of how Selenium CDP Listeners can be utilized to extract a plethora of data from network responses during a testing scenario. Consider a common situation where a web page is making multiple network requests for resources. By setting up listeners, we can intercept and extract a range of valuable information.

public class NetworkActivity {

    public static void main(String[] args) {
        
        WebDriverManager.chromedriver().setup();
        ChromeDriver driver  = new ChromeDriver();
        
        //create session
        DevTools devTools = driver.getDevTools();
        
        devTools.createSession();
        
        //send cmd to CDP methods> cdp method will invoke and get access to chrome devtools
        
        //enable network
        devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
        
        
        //event fires > listeners will capture the event
        
        //request event 
        devTools.addListener(Network.requestWillBeSent(), request->{
            
            Request req = request.getRequest();
            System.out.println(req.getUrl());
            System.out.println(req.getHeaders());
            
        });
        
        //response event
        devTools.addListener(Network.responseReceived(), response ->{
            
            org.openqa.selenium.devtools.v120.network.model.Response resp = response.getResponse();
            System.out.println(resp.getUrl());
            System.out.println(resp.getStatus());
            System.out.println(response.getRequestId());
             
            
        });
        
        //perform test
        driver.get("https://rahulshettyacademy.com/angularAppdemo");
        
        driver.findElement(By.linkText("Library")).click();

        driver.quit();
        
    }
}

In this example, we enable the Network domain and listen for the responseReceived and ‘requestWillBeSent’ events. Upon receiving the event, we extract the status code, request ID, URLs, and request header data from the response. Below are some functions we have used above

1. ‘Network.enable’ (Network Enablement): Sends a command to enable the network domain in Chrome DevTools, allowing interception of network events. It accepts 3 parameters: maxTotalBufferSize(integer), maxResourceBufferSize(integer), and maxPostDataSize(integer) respectively.
2. ‘Network.requestWillBeSent’ Event Listener: You can listen for the event triggered when a network request is about to be sent. Capture and print the URL and headers of the outgoing request.
3. ‘Network.responseReceived’ Event Listener: You can listen for the event triggered when a network response is received. Capture and print the URL, status, and request ID associated with the response.

We can further customize our actions based on the code.

Real-world Applications

Understanding network status codes in real time is invaluable for comprehensive testing. It enables us to identify potential bottlenecks, optimize resource loading, and ensure that the web application behaves as expected under different network conditions.

You can refer to official documentation for more details on various methods and events: Chrome DevTools Protocol

Conclusion

As we conclude this exploration into harnessing network responses with Selenium CDP Listeners, we unlock a new dimension of insight in web testing. The capability to extract and analyze a diverse range of data points from network responses enhances the precision and effectiveness of our testing strategies. Stay tuned for more revelations in Selenium CDP integration as we continue to decode the intricacies of web interactions. Happy testing!