In this post, we will learn
BrowserMob Proxy Selenium Java
test automation. The execution of web pages generally involves three tasks:
fetching resources
,
page layout and rendering
, and
JavaScript execution.
Apart from these, sometimes websites need to track user behaviors so they need many third-party integrations and this
may affect
page load time
. Also, we can have different problems like
javascript
, instability, etc. We can filter all these things using
BroserMob Proxy
.
We will go over
how can we get performance-related data from selenium test scenarios & integrate them with browsermob proxy
.
BrowserMob Proxy Outline
Page Load & Bandwith & Latency Relation
What is BrowserMob Proxy?
BrowserMob Features
What is HAR?
Creating Profile
Creating a Selenium project using BrowserMob
Chrome Developer Tools
Page Load & Bandwidth & Latency Relation
(Picture from High-Performance Browser Networking)
Latency:
The time from the source sending a packet to the destination receiving it.
Bandwidth:
Maximum throughput of a logical or physical communication path.
On that page its definition is as follows: BrowserMob Proxy allows you to manipulate HTTP requests and responses, capture HTTP content, and export performance data as a
HAR file
. BMP works well as a standalone proxy server, but it is especially useful when embedded in Selenium tests.
Features of BrowserMob Proxy
The proxy is programmatically controlled via a REST interface or by being embedded directly inside Java-based programs and unit tests. It captures performance data in the HAR format. In addition, it can actually control HTTP traffic, such as:
Black-listing and white-listing certain URL patterns
Simulating various bandwidth and latency
Remapping DNS lookups
Flushing DNS caching
Automatic BASIC authorization
What is HAR?
The HAR format is based on JSON, as described in
RFC 4627
.
Har
is an open format for exporting HTTP tracing information called
HTTP Archive (HAR).
The data is stored as a JSON document.
What kind of information we can get from har file:
DNS information load time
E
ach object requested (load time)
Connection time to the server
From server to a browser (load time)
The object is blocked or not
Some of HAR object types are log, creator, browser, pages, page timings, etc.
We can check har file results using different tools:
You can check the new browser mob proxy interface and its methods from this link
Chrome Developer Tools
Copy All as HAR: Copies all resources to the system clipboard as HAR data. A HAR file contains a JSON data structure.
Save as HAR with Content: Saves all network data to a HAR file along with each page resource.
Chrome Developer Tools, go to the Network tab, and make sure you’ve selected the Preserve log checkbox, otherwise, the log is reset when you change the page.
Unfortunately with the latest selenium versions create problems. I tried all options to fix the problem but I could not. If you find a solution to run BrowserMobProxy with Selenium’s latest versions, please write a comment about that solution or you can create PR in github repo.
When I tried convert Har to Jmx through blazemeter then it’s showing me “Fatal error during conversion:
Failed to validate json file SeleniumEasy.har: check file format”.
Also getting below error logs:
2017/04/25 04:40:08 INFO – com.blazemeter.utils.Configuration: Configuration was loaded…
2017/04/25 04:40:08 INFO – com.blazemeter.Main: Log file=/var/cache/tomcat8/temp/7nhbj8kaarnlk3jemcnttusson9067326786491721299/SeleniumEasy.log
2017/04/25 04:40:08 INFO – com.blazemeter.Main: Input file=/var/cache/tomcat8/temp/7nhbj8kaarnlk3jemcnttusson9067326786491721299/SeleniumEasy.har
2017/04/25 04:40:08 INFO – com.blazemeter.Main: Output file=/var/cache/tomcat8/temp/7nhbj8kaarnlk3jemcnttusson9067326786491721299/SeleniumEasy.jmx
2017/04/25 04:40:08 INFO – com.blazemeter.utils.Utils: Trying to detect input file type vie mime_type….
2017/04/25 04:40:08 INFO – com.blazemeter.utils.Utils: Detected input file type=text/html
2017/04/25 04:40:08 ERROR – com.blazemeter.Main: Failed to start converting: com.github.fge.jsonschema.core.exceptions.ProcessingException: fatal: Failed to find jsonSchema for /var/cache/tomcat8/temp/7nhbj8kaarnlk3jemcnttusson9067326786491721299/SeleniumEasy.har
level: “fatal”
at com.blazemeter.utils.JsonFormatDetector.converterType(JsonFormatDetector.java:187)
at com.blazemeter.utils.Utils.converterType(Utils.java:78)
at com.blazemeter.utils.Utils.converter(Utils.java:187)
at com.blazemeter.Main.main(Main.java:83)
at com.blazemeter.Conversion$ConversionTask.call(Conversion.java:56)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
It is better to ask Blazemeter support. :( This part has a problem. It can not find json schema.
com.github.fge.jsonschema.core.exceptions.ProcessingException: fatal: Failed to find jsonSchema for /var/cache/tomcat8/temp/7nhbj8kaarnlk3jemcnttusson9067326786491721299/SeleniumEasy.har
level: “fatal”
Thank you for your helpful Tutorial, I did as you explain but I get this it fail up in the first Test,
AILED CONFIGURATION: @BeforeClass setup
java.lang.NoSuchMethodError: com.google.common.io.Closeables.closeQuietly(Ljava/io/Closeable;)V
at org.openqa.selenium.firefox.internal.ClasspathExtension.writeTo(ClasspathExtension.java:62)
at org.openqa.selenium.firefox.FirefoxProfile.installExtensions(FirefoxProfile.java:527)
at org.openqa.selenium.firefox.FirefoxProfile.layoutOnDisk(FirefoxProfile.java:505)
at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:75)
at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:149)
at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:72)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:128)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:87)
at com.proxy.BrowserMobProxy.MobProxy.setup(MobProxy.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:551)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:138)
at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:175)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:107)
at org.testng.TestRunner.privateRun(TestRunner.java:768)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1188)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1113)
at org.testng.TestNG.run(TestNG.java:1025)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:132)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:230)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:76)
Thank your for yout answer I fixed this problem then I get exception on @Before test, The problem is surely in the seleniumCapabilities, it can’t accept this constructor driver = new FirefoxDriver(seleniumCapabilities);
FAILED CONFIGURATION: @BeforeClass setup
org.openqa.selenium.SessionNotCreatedException: InvalidArgumentError: Expected [object Undefined] undefined to be an integer
Build info: version: ‘3.4.0’, revision: ‘unknown’, time: ‘unknown’
System info: host: ‘LAPTOP-CE1IQ7E0’, ip: ‘192.168.1.95’, os.name: ‘Windows 10’, os.arch: ‘amd64’, os.version: ‘10.0’, java.version: ‘1.8.0_121’
Driver info: driver.version: FirefoxDriver
remote stacktrace: stack backtrace:
0: 0x48916f –
1: 0x489f59 –
2: 0x439ced –
3: 0x4473ea –
4: 0x445128 –
5: 0x41dbb1 –
6: 0x407997 –
7: 0x6bbb39 –
8: 0x415b39 –
9: 0x6b6043 –
10: 0x7ffcdc038364 – BaseThreadInitThunk
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.openqa.selenium.remote.W3CHandshakeResponse.lambda$new$0(W3CHandshakeResponse.java:57)
at org.openqa.selenium.remote.W3CHandshakeResponse.lambda$getResponseFunction$2(W3CHandshakeResponse.java:104)
at org.openqa.selenium.remote.ProtocolHandshake.lambda$createSession$22(ProtocolHandshake.java:365)
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.util.Spliterators$ArraySpliterator.tryAdvance(Unknown Source)
at java.util.stream.ReferencePipeline.forEachWithCancel(Unknown Source)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.util.stream.FindOps$FindOp.evaluateSequential(Unknown Source)
at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.util.stream.ReferencePipeline.findFirst(Unknown Source)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:368)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:159)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:142)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:82)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:637)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:250)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:236)
at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:137)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:191)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:108)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:137)
at com.proxy.BrowserMobProxy.MobProxy.setup(MobProxy.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:551)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:138)
at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:175)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:107)
at org.testng.TestRunner.privateRun(TestRunner.java:768)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1188)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1113)
at org.testng.TestNG.run(TestNG.java:1025)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:132)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:230)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:76)
===============================================
Default suite
Total tests run: 1, Failures: 0, Skips: 1
Configuration Failures: 1, Skips: 1
===============================================
[TestNG] Time taken by [TestListenerAdapter] Passed:0 Failed:0 Skipped:0]: 22 ms
[TestNG] Time taken by org.testng.reporters.XMLReporter@4909b8da: 13 ms
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@60addb54: 87 ms
[TestNG] Time taken by org.testng.reporters.jq.Main@614c5515: 58 ms
[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@6df97b55: 6 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter@6537cf78: 5 ms
//WebDriver
WebDriver driver = new PhantomJSDriver(capabilities);
driver.get(“https://www.google.co.in/”);
//HAR
Har har = server.getHar();
FileOutputStream fos = new FileOutputStream(“C:\\HAR-Information.har”);
har.writeTo(fos);
server.stop(); Reply
I want to you browsermobproxy in TESTNG such that in @BeforeSuite annotation/method , it will get started and n @AfterSuite annotation/ method, it will get closed.
I want to run Browsermobproxy tool in windows 7 and Mac with selenium cucumber. please some one help how to setup Browsemobproxy and integrate with selenium cucumber
Hi guys, i’m currently using the browsermob, and i’m tracking the mixed content URL but the browsermob did not return any mixed content. mixed content are url that was is http. Right now, our website was migrated to https. Please response, thanks
hi onur baskirt
can u please write an code that asserts two values in network tap using java in selenium .
i have an requirement like ,
consider two strings(json format) that are in different links , we need to compare them whether they are equal or not.
Nani, I really would like to help but nowadays I am super busy. If I will have time, I will help. But I will also search this on the internet. But when I read your request, I think it is better to do this with Rest-Assured. As I understand, you are trying to compare an API result and that result format is JSON. With rest-assured you can do this much easier. Please check here: http://www.swtestacademy.com/api-testing-with-rest-assured/
Reply
This is not working could u please check and fix.
I have a scenario here to get help from you. After few navigation on UI I need to get the some API response from the browser network and do verify the some value in the response.
How to get the specific API response from the har logs in java.
is there any other to get the required API response from the browser network tab.?