New interface org.apache.jmeter.engine.util.ConfigMergabilityIndicator has been introduced to tell whether a ConfigTestElement can be merged in Sampler (see Bug 53042 New interface org.apache.jmeter.protocol.http.proxy.SamplerCreator to allow plugging HTTP based samplers that differ from default HTTP Samplers through Proxy during Recording Phase (see Bug 52674 The Once Only controller behaves correctly under a Thread Group or Loop Controller, but otherwise its behaviour is not consistent (or clearly specified). Listeners don't show iteration counts when a If Controller has a condition which is always false from the first iteration (see Bug 52496 A workaround is to add a sampler at the same level as (or superior to) the If Controller. For example a Test Action sampler with 0 wait time (which doesn't generate a sample), or a Debug Sampler with all fields set to False (to reduce the sample size). Incompatible changes When doing replacement of User Defined Variables, Proxy will not substitute partial values anymore when "Regexp matching" is used. It will use Perl 5 word matching ("\b") In User Defined Variables, Test Plan, HTTP Sampler Arguments Table, Java Request Defaults, JMS Sampler and Publisher, LDAP Request Defaults and LDAP Extended Request Defaults, rows with empty Name and Value are no more saved. Upgraded to rhino 1.7R3 (was js-1.7R2.jar). Note: the Maven coordinates for the jar were changed from rhino:js to org.mozilla:rhino. This does not affect JMeter directly, but might cause problems if using JMeter in a Maven project with other code that depends on an earlier version of the Rhino Javascript jar. The menu item Options / Choose Language now changes all the displayed text to the new language provided all messages are translated. You can help on this by translating into your language. Legacy JMX and JTL Avalon format support restored Support for reading/writing the original Avalon XML format of JMX (script) and JTL (sample result) files was dropped in JMeter version 2.4. JMeter can now read the Avalon format files again, however there is no support for saving files in the old format. JMeter jars available from Maven repository JMeter jars are now available from Maven repository. Known bugs The Include Controller has some problems in non-GUI mode (see Bugs 40671, 41286, 44973, 50898). In particular, it can cause a NullPointerException if there are two include controllers with the same name. The Once Only controller behaves correctly under a Thread Group or Loop Controller, but otherwise its behaviour is not consistent (or clearly specified). Listeners don't show iteration counts when a If Controller has a condition which is always false from the first iteration (see Bug 52496 A workaround is to add a sampler at the same level as (or superior to) the If Controller. For example a Test Action sampler with 0 wait time (which doesn't generate a sample), or a Debug Sampler with all fields set to False (to reduce the sample size). Incompatible changes JMeter versions since 2.1 failed to create a container sample when loading embedded resources. This has been corrected; can still revert to the Bug 51939 behaviour by setting the following property: httpsampler.separate.container=false TCP Sampler handles SocketTimeoutException, SocketException and InterruptedIOException differently since 2.6, when these occurs, Sampler is marked as failed. Bug 52161 - Enable plugins to add own translation rules in addition to upgrade.properties. Loads any additional properties found in META-INF/resources/org.apache.jmeter.nameupdater.properties files Add option to change the pause time (default 2000ms) in the daemon thread which checks for successful JVM exit. The thread is not now started unless the pause time is greater than 0. The Once Only controller behaves correctly under a Thread Group or Loop Controller, but otherwise its behaviour is not consistent (or clearly specified). The If Controller may cause an infinite loop if the condition is always false from the first iteration. A workaround is to add a sampler at the same level as (or superior to) the If Controller. For example a Test Action sampler with 0 wait time (which doesn't generate a sample), or a Debug Sampler with all fields set to False (to reduce the sample size). The menu item Options / Choose Language does not change all the displayed text to the new language. [The behaviour has improved, but language change is still not fully working] To override the default local language fully, set the JMeter property "language" before starting JMeter. Incompatible changes The HttpClient4 and Commons HttpClient 3.1 samplers previously used a retry count of 3. This has been changed to default to 1, to be compatible with the Java implementation. The retry count can be overridden by setting the relevant JMeter property, for example: httpclient4.retrycount=3 httpclient3.retrycount=3 If Controller - Catches a StackOverflowError when a condition returns always false (after at least one iteration with return true) See Bug 50618 The HTTP implementation can now be selected at run-time, and JMeter now also supports Apache HttpComponents HttpClient 4.x. Note that Commons HttpClient 3.1 is no longer actively developed, and support may be removed from JMeter in a future release. The menu item Options / Choose Language does not change all the displayed text to the new language. [The behaviour has improved, but language change is still not fully working] To override the default local language fully, set the JMeter property "language" before starting JMeter. Incompatible changes Unsupported methods are no longer converted to GET by the Commons HttpClient sampler. Bug 51268 - HTTPS request through an invalid proxy causes NullPointerException and does not show in result tree. Rather than delegating to the JMeter thread handler for "unexpected" failures, ensure all Exceptions generate a sample error. System.nanoTime() tends to drift relative to System.currentTimeMillis(). Change SampleResult to recalculate offset each time. Also enable reversion to using System.currentTimeMillis() only. View Results Tree - Add new size fields: response headers and response body (in bytes) - derived from Bug 43363 Size Assertion - Add response size scope (full, headers, body, code, message) - derived from Bug 43363 UDP shutdown can now use a range of ports, from jmeterengine.nongui.port=4445 to jmeterengine.nongui.maxport=4455, allowing multiple JMeter instances on the same host without needing to change the port property. JMS samplers have been extensively reworked, and should no longer lose messages. Correlation processing is improved. JMS Publisher and Subscriber now support both Topics and Queues. The menu item Options / Choose Language does not change all the displayed text to the new language. [The behaviour has improved, but language change is still not fully working] To override the default local language fully, set the JMeter property "language" before starting JMeter. Incompatible changes HTTP Redirect now defaults to "Follow Redirects" rather than "Redirect Automatically". This is to enable JMeter to track cookies that may be sent during redirects. This does not affect existing test plans; it only affects the default for new HTTP Samplers. The constructor public SampleResult(SampleResult res) has been changed to become a true "copy constructor". It no longer calls addSubResult(). This may possibly affect some 3rd party add-ons. Bug fixes HTTP Samplers and Proxy JMS Publisher and Subscriber now support both Topics and Queues. Added read Timeout to JMS Subscriber. General clean-up of JMS code. Listeners using SamplingStatCalculator have much reduced memory needs as the Sample cache has been moved to the new CachingStatCalculator class. In particular, Aggregate Report can now handle large numbers of samples. The menu item Options / Choose Language does not change all the displayed text to the new language. [The behaviour has improved, but language change is still not fully working] To override the default local language fully, set the JMeter property "language" before starting JMeter. Bug fixes HTTP Samplers and Proxy The handling of test closedown is much improved. The gradual "Shutdown" command now waits until all threads have stopped, and does not report an error if threads don't stop within 5 seconds. The immediate "Stop" command can now be used if "Shutdown" takes too long. Also the immediate "Stop" command is able to interrupt samplers which support the new Interruptible interface (e.g. HTTP and SOAP, FTP). This allows immediate completion of pending responses. Non-GUI mode tests can also now be sent a "Shutdown" or "Stop" message. Test Action now supports a "Stop Now" action, as do the Thread Group Result Status Action Handler Post Processor elements. HTTP Cookie handling is improved, and HTTP POST can now use variable file names correctly. HTTP, SOAP/XML-RPC and WebService(SOAP) sampler character encodings updated to be more consistent. HTTP Samplers now support connection and response timeouts (requires JVM 1.5 for the HTTP Java sampler). Together with the closedown improvements described above, this should avoid most cases where a test run hangs. Multiple Header Manager elements are now supported for a single HTTP sampler. The Proxy Server is improved, and no longer stores "Host" headers by default. __char() function: allows arbitrary Unicode characters to be entered in fields. __unescape() function: allows Java-escaped strings to be used. _unescapeHtml() function: decodes Html-encoded text. __escapeHtml() function: encodes text using Html-encoding. A reference to a missing function - e.g. ${__missing(a)} - is now treated the same as a missing variable. Previously the function name - and leading { - were dropped. This makes it easier to debug test plans. Some Assertions can now be applied to sub-samples as well as (or instead of) just the parent sample. There is a new Random Variable Configuration element. Various I18N changes have been made; language change works better (though not perfect yet). There are improved French translations as well as new Polish and Brazilian Portugese translations. The menu item Options / Choose Language does not change all the displayed text to the new language. [The behaviour has improved, but language change is still not fully working] To override the default local language fully, set the JMeter property "language" before starting JMeter. Incompatible changes When loading sample results from a file, previous results are no longer cleared. This allows one to merge multiple files. If the previous behaviour is required, use the menu item Run/Clear (Ctrl+Shift+E) or Run/Clear All (Ctrl+E) before loading the file. The test elements "Save Results to a file" and "Generate Summary Results" are now shown as Listeners. They were previously shown as Post-Processors, even though they are implemented as Listeners. The Cookie Manager no longer saves incoming cookies as variables by default. To save cookies as variables, define the property "CookieManager.save.cookies=true". Also, cookies names are prefixed with "COOKIE_" before they are stored (this avoids accidental corruption of local variables) To revert to the original behaviour, define the property "CookieManager.name.prefix= " (one or more spaces). The above changes only affect the icons that are displayed and the locations in the GUI pop-up menus. They do not affect test plans or test behaviour. The PreProcessors are now invoked directly by the JMeterThread class, rather than by the TestCompiler#configureSampler() method. (JMeterThread handles the PostProcessors). This does not affect test plans or behaviour, but could perhaps affect 3rd party add-ons (very unlikely). The While controller now trims leading and trailing spaces from the condition value before it is compared with LAST, blank or false. The following deprecated methods were removed from JOrphanUtils: booleanToString(boolean) and valueOf(boolean). Java 1.4+ has these methods in the Boolean class. Fix incorrect GUI classifications: "Save Results to a file" and "Generate Summary Results" are now shown as Listeners. "Counter" is now shown as a Configuration element. The Menu items Stop and Shutdown now behave better. Shutdown will now wait until all threads exit. In GUI mode it can be cancelled and Stop run instead. Stop now reports if some threads will not exit, and exits if running in non-GUI mode Version 2.3.1 changed the way binary and text content types were determined as far as the View Results Tree Listener was concerned: originally everything except "image/" content types were considered text, but 2.3.1 introduced a check for specific content types. This has caused problems, as several popular types were omitted and these were no longer shown by default in the Response tab. Rather than try to list all the possible text types, JMeter now just checks for the following binary types: JMeter 2.3.1 introduced a bug in the Cookie Manager - if "Clear Cookie each iteration" was selected, all threads would see the same cookies. This bug has been corrected. Improvements The Proxy server can now record binary requests. By default the content types application/x-amf and application/x-java-serialized-object will be treated as binary and saved in a file. To change the content types, update the property proxy.binary.types The CSV Dataset configuration element has new file sharing options: per thread group, per thread, per identifier. This allows for more flexible file processing, e.g. each thread can process the same data in the same order. Switch Controller now works properly with functions and variables, and the condition can now be a name instead of a number. Simple Controller now works properly under a While Controller CSV fields in JTL files can now contain delimiters. CSV and XML files can now contain additional variables (define the JMeter property sample_variables Response Assertion can now match on substrings (i.e. not regular expression). Regex extractor can operate on variables. XPath processing is improved; Tidy errors are handled better. Save Table Data buttons added to Summary and Aggregate reports to allow easy saving of the calculated data. HTTP samplers can now save just the MD5 hash of responses, rather than the entire response. As a special case, if the HTTP Sampler path starts with "http://" or "https://" then this is used as the full URL, overriding the host and port fields. The HTTP Samplers can now POST multiple files. Webservice(SOAP) Sampler can now load local WSDL files using the "file:" protocol. View Results Tree Listener now uses Tidy to display XML. This should allow more content to be displayed succesfully. It also avoids the need to download remote DTD files, which can slow the rendering considerably. Various improvements to the BSF Sampler: now supports Jexl, and Javascript bug works properly. Added BSF PreProcessor, PostProcessor and Assertion test elements. All now have access to "props" JMeter Properties object. Number of classes loaded in non-GUI mode is much reduced. Known bugs The Include Controller has some problems in non-GUI mode. In particular, it can cause a NullPointerException if there are two include controllers with the same name. Once Only controller behaves OK under a Thread Group or Loop Controller, but otherwise its behaviour is not consistent (or clearly specified). The menu item Options / Choose Language does not change all the displayed text to the new language. To override the default local language, set the JMeter property "language" before starting JMeter. Incompatible changes To reduce the number of classes loaded in non-GUI mode, Functions will only be found if their classname contains the string '.functions.' and does not contain the string '.gui.'. All existing JMeter functions conform to this restriction. To revert to earlier behaviour, comment or change the properties classfinder.functions.* in jmeter.properties. Synchronization has been removed from the RunningSample class (it was not fully threadsafe anyway). Developers of 3rd party add-ons that use the class may need to synchronize access. The BeanShell and BSF component documentation made some incorrect references to the "SampleResponse" object; this has been corrected to "SampleResult" The Proxy spoof function was broken in 2.3; it has been fixed. Spoof now supports an optional parameter to limit spoofing to particular URLs. This is useful for HTTPS pages that have insecure content - e.g. images/stylesheets may be accessed using HTTP. Spoofed responses now drop the default port (443) from https links to make them work better. Ignored proxy samples are now visible in Listeners - the label is enclosed in [ and ] as an indication. Proxy documentation has been improved. GUI changes The Add menus show element types in the order in which they are processed - see Test Plan Execution Order It is no longer possible to add test elements to inappropriate parts of the tree - e.g. samplers cannot be added directly under a test plan. This also applies to Paste and drag and drop. Command-line option -G can now be used to define properties for remote servers. Option -X can be used to stop a remote server after a non-GUI run. Server can be set to automatically exit after a single test (set property server.exitaftertest=true). Other enhancements JMeter startup no longer loads as many classes; this should reduce memory requirements. Parameter and file support added to all BeanShell elements. Javascript function now supports access to JMeter objects; Jexl function always did have access, but the documentation has now been included. New functions __eval() and __evalVar() for evaluating variables. CSV files with the correct header column names are now automatically recognised when loaded. There is no need to configure the properties. The hostname can now be saved in CSV and XML output files. New "Successes only" option added when saving result files. Errors / Successes only option is now supported when loading XML and CSV files. JMeter server now creates the RMI registry by default. If the RMI registry has already been started externally, this will generate a warning message, but the server will continue. This should not affect JMeter testing. However, if you are also using the RMI registry for other applications there may be problems. For example, when the JMeter server shuts down it will stop the RMI registry. Also user-written command files may need to be adjusted (the ones supplied with JMeter have been updated). To revert to the earlier behaviour, define the JMeter property: server.rmi.create=false The Proxy server removes If-Modified-Since and If-None-Match headers from generated Header Managers. To revert to the previous behaviour, define the property proxy.headers.remove with no value Fix SamplingStatCalculator so it no longer adds elapsed time to endTime, as this is handled by SampleResult. This corrects discrepancies between Summary Report and Aggregate Report throughput calculation. The Proxy server removes If-Modified-Since and If-None-Match headers from generated Header Managers by default. To change the list of removed headers, define the property proxy.headers.remove as a comma-separated list of headers to remove The JMeter remote server does not support multiple concurrent tests - each remote test should be run in a separate server. Otherwise tests may fail with random Exceptions, e.g. ConcurrentModification Exception in StandardJMeterEngine. Bug 43168 The default HTTP Request (not HTTPClient) sampler may not work for HTTPS connections via a proxy. This appears to be due to a Java bug, see Bug 39337 To avoid the problem, try a more recent version of Java, or switch to the HTTPClient version of the HTTP Request sampler. Transaction Controller parent mode does not support nested Transaction Controllers. Doing so may cause a Null Pointer Exception in TestCompiler. Thread active counts are always zero in CSV and XML files when running remote tests. The property file_format.testlog=2.1 is treated the same as 2.2. However JMeter does honour the 3 testplan versions. Variables and functions do not work in Listeners in client-server (remote) mode so they cannot be used to name log files in client-server mode. CSV Dataset variables are defined after configuration processing is completed, so they cannot be used for other configuration items such as JDBC Config. Bug 40394 Summary of changes (for more details, see below) Some of the main enhancements are: N.B. The javax.net.ssl properties have been moved from jmeter.properties to system.properties, and will no longer work if defined in jmeter.properties. SSL session contexts are now created per-thread, rather than being shared. This generates a more realistic load for HTTPS tests. The change is likely to slow down tests with many SSL threads. The original behaviour can be enabled by setting the JMeter property: https.sessioncontext.shared=true The LDAP Extended Sampler now uses the same panel for both Thread Bind and Single-Bind tests. This means that any tests using the Single-bind test will need to be updated to set the username and password. Bug 41140 : JMeterThread behaviour was changed so that PostProcessors are run in forward order (as they appear in the test plan) rather than reverse order as previously. The original behaviour can be restored by setting the following JMeter property: The HTTP Authorisation Manager now has extra columns for domain and realm, so the temporary work-round of using '\' and '@' in the username to delimit the domain and realm has been removed. The clear() method was defined in the following interfaces: Clearable, JMeterGUIComponent and TestElement. The methods serve different purposes, so two of them were renamed: the Clearable method is now clearData() and the JMeterGUIComponent method is now clearGui(). 3rd party add-ons may need to be rebuilt. Calulator and SamplingStatCalculator classes no longer provide any formatting of their data. Formatting should now be done using the jorphan.gui Renderer classes. The time stamp is now set to the sampler start time (it was the end). To revert to the previous behaviour, change the property sampleresult.timestamp.start to false (or comment it) The JMX output format has been simplified and files are not backwards compatible The JMeter.BAT file no longer changes directory to JMeter home, but runs from the current working directory. The jmeter-n.bat and jmeter-t.bat files change to the directory containing the input file. The GUI now expands the tree by default when loading a test plan. This can be disabled by setting the JMeter property onload.expandtree=false Enhanced JUnitSampler so that by default assert errors and exceptions are not appended to the error message. Users must explicitly check append in the sampler Sample Result converter saves response code as "rc". Previously it saved as "rs" but read with "rc"; it will now also read with "rc". The XSL stylesheets also now accept either "rc" or "rs" Thread Group Gui now has an extra pane to specify what happens after a Sampler error: Continue (as now), Stop Thread or Stop Test. This needs to be extended to a lower level at some stage. Remote samples can be cached until the end of a test by defining the property hold_samples=true when running the server. More work is needed to be able to control this from the GUI Extra Proxy Server options: Create all samplers with keep-alive disabled Add Separator markers between sets of samples Add Response Assertion to first sampler in each set View Results Tree now displays more information, including name of request (matching the name in the test tree) and full request and POST data. New Http sampling that automatically parses HTML response for images to download, and includes the downloading of these images in total time for request (Neth [email protected]) HTTP responses from server can be parsed for links and forms, and dynamic data can be extracted and added to test samples at run-time (documented) Abstracted out the concept of the Sampler, SamplerController, and TestSample. A Sampler represents code that understands a protocol (such as HTTP, or FTP, RMI, SMTP, etc..). It is the code that actually makes the connection to whatever is being tested. A SamplerController represents code that understands how to organize and run a group of test samples. It is what binds together a Sampler and its test samples and runs them. A TestSample represents code that understands how to gather information from the user about a particular test. For a website, it would represent a URL and any information to be sent with the URL. Created the opportunity to create URL aliasing from the properties file as well as the ability to associate aliases to URL sequences instead of single URLs (SM) Thanks to Simon Chatfield for the very nice suggestions and code examples.