BrowserMobProxy configuration

General Proxy configuration

Configuring the Driver for the Proxy

Then do your ’test’ stuff

Checking HTTP Traffic

Closing the BrowserMobProxy

Full Source

To capture HTTP Messages and network traffic with Selenium WebDriver we need to use an HTTP proxy, and configure the browser to direct all traffic through the proxy.

For this example I’m using an in code library as the HTTP proxy because that makes control of the test eaasier.

But this will work with any external proxy, you just have to start the proxy before you start the test.

HTTP Proxy

I’m using BrowserMob Proxy.

https://github.com/lightbody/browsermob-proxy

http://bmp.lightbody.net/

Which I add to my pom.xml as a dependency:

    <dependency>
        <groupId>net.lightbody.bmp</groupId>
        <artifactId>browsermob-core</artifactId>
        <version>2.1.5</version>
        <scope>test</scope>
    </dependency>

Page Using XHTTP Requests

For this example I am using:

https://testpages.herokuapp.com/styled/sync/xhttp-messages.html

Which, if you look in the Network traffic tab of your Browser Dev Tools, makes some XHTTP Requests (XHR) to update the page.

With a proxy we should be able to capture that traffic and extend the reach of our assertions.

BrowserMobProxy configuration

Start the proxy:

    proxyServer = new BrowserMobProxyServer();
    proxyServer.start();
    // capture content as a HAR (HTTP Archive)
    // to process when test is complete
    proxyServer.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT,
                                      CaptureType.RESPONSE_CONTENT);
    proxyServer.newHar();

And create the proxy config:

    final Proxy proxyConfig = ClientUtil.createSeleniumProxy(proxyServer);

General Proxy configuration

To configure a proxy in general we configure the proxy details:

    String proxyDetails = "127.0.0.1:8080";
    final Proxy proxyConfig = new Proxy().
                                setHttpProxy(proxyDetails).
                                setSslProxy(proxyDetails);

Configuring the Driver for the Proxy

And use those to configure the driver:

    final FirefoxOptions firefoxOptions = new FirefoxOptions();
    firefoxOptions.setProxy(proxyConfig);
    driver = new FirefoxDriver(firefoxOptions);

For Chrome I set the additional config for insecure certs:

    final ChromeOptions options = new ChromeOptions();
    options.setProxy(proxyConfig);
    options.setAcceptInsecureCerts(true);
    driver = new ChromeDriver(options);

Then do your ’test’ stuff

The test should run as normal, but now all traffic is going through the proxy.

Checking HTTP Traffic

For an external proxy you will then review the traffic externally in the proxy.

For BrowserMobProxy, because it is running in memory we can process the HTTP messages as part of the test.

    final Har httpMessages = proxyServer.getHar();
    for(HarEntry httpMessage : httpMessages.getLog().getEntries()){
        // check no errors on the XHR requests
        if(httpMessage.getRequest().getUrl().contains("/messageset")) {
            Assertions.assertEquals(200, httpMessage.getResponse().getStatus());

BrowserMobProxy has a lot more configuration and usage approaches, but the above is the simplest use of gathering traffic as you test.

Closing the BrowserMobProxy

    proxyServer.abort();

Full Source

The full source for this is in my Webdriver Java FAQs project:

https://github.com/eviltester/seleniumWebDriverCompendium/tree/main/webdriverjavafaqs

Specifically:

https://github.com/eviltester/seleniumWebDriverCompendium/tree/main/webdriverjavafaqs/src/test/java/proxies/HowToCaptureNetworkTraffic

If you want to learn how to use Selenium WebDriver with Java then check out our online courses .

Join The Evil Tester Patreon Community

Free training courses and ebooks (sold for over $200).

Learn more about EvilTester on Patreon

Support our work and gain access to hints, tips, and prompts for improving your Software Development skills. Regular updates, multiple times a week for as little as $1 per month.

Patreon Members Login for exclusive courses and ebooks.

Modelling Software Testing for Naming and Understanding

Kubernetes kubectl cheat sheet notes

Updated Github Repos and Released Conference Talk

Recruit And interview people. Do not just fill roles

Testing The Triangle Application

On Modelling

June 2023 EvilTester.com and Patreon Content Summary

Episode 020 - The Test Automation Pyramid Episode - The Evil Tester Show