The Need for the Creation of the STS Plugin

From a Web Application in Tomcat

The idea of having a server to manage the dataset was born during the performance tests of the income tax declaration application of the French Ministry of Public Finance in 2012. The dataset consisted of millions of lines to simulate tens of thousands of people who filled out their income tax return form per hour and there were a dozen injectors to distribute the injection load of a performance shot. The dataset was consumed, that is to say, once the line with the person's information was read or consumed, we could no longer take the person's information again.

The management of the dataset in a centralized way had been implemented with a Java web application (war) running in Tomcat . Injectors requesting a row of the dataset from the web application.

To a Plugin for Apache JMeter

The need to have centralized management of the dataset, especially with an architecture of several JMeter injectors, was at the origin of the creation in 2014 of the plugin for Apache JMeter named HTTP Simple Table Server or STS. This plugin takes over the features of the dedicated application for Tomcat mentioned above but with a lighter and simpler technical solution based on the NanoHTTPD library.

Manage the Dataset With HTTP Simple Table Server (STS)

Adding Centralized Management of the Dataset

Performance tests with JMeter can be done with several JMeter injectors or load generators (without interface, on a remote machine) and a JMeter controller (with user interface or command line interface on the local machine). JMeter script and properties are sent by RMI protocol to injectors. The results of the calls are returned periodically to the JMeter controller.

However, the dataset and CSV files are not transferred from the controller to the injectors.

It is natively not possible with JMeter to read the dataset randomly or in reverse order. However, there are several external plugins that allow you to randomly read a data file, but not in a centralized way.

It is natively not possible to save data created during tests such as file numbers or new documents in a file. The possibility of saving values can be done with the Groovy script in JMeter, but not in a centralized way if you use several injectors during the performance test.

The main idea is to use a small HTTP server to manage the dataset files with simple commands to retrieve or add data lines to the data files. This HTTP server can be launched alone in an external program or in the JMeter tool.

The HTTP server is called the " S imple T able S erver" or STS. The name STS is also a reference to the Virtual Table Server (VTS) program of LoadRunner with different functionalities and a different technical implementation, but close in the use cases.

STS is very useful in a test with multiple JMeter injectors, but it also brings interesting features with a single JMeter for performance testing.

Some Possibilities of Using the HTTP Simple Table Server

Reading Data in a Distributed and Collision-Free Way

Some applications do not tolerate that 2 users connect with the same login at the same time. It is often recommended not to use the same data for 2 users connected with the same login at the same time to avoid conflicts on the data used.

The STS can easily manage the dataset in a distributed way to ensure that logins are different for each virtual user at a given time T of the test. A dataset with logins/passwords with a number of lines greater than the number of active threads at a given time is required.

We manually load at the start of the STS or by script the logins/password file, and the injectors ask the STS for a line with login/password by the command READ , KEEP=TRUE , and READ_MODE=FIRST .

We can consider that the reading of the dataset is done in a circular way.

Reading Single-Use Data

The dataset can be single-use, meaning that it is used only once during the test. For example, people who register on a site can no longer register with the same information because the system detects a duplicate. Documents are awaiting validation by an administrator. When the documents are validated, they are no longer in the same state and can no longer be validated again.

To do this, we will read a data file in the memory of the HTTP STS and the virtual users will read by deleting the value at the top of the list. When the test is stopped, we can save the values that remain in memory in a file (with or without a timestamp prefix) or let the HTTP STS run for another test while keeping the values still in memory.

Producers and Consumers of a Queue

With the STS, we can manage a queue with producers who deposit in the queue and consumers who consume the data of the queue. In practice, we start a script with producers who create data like new documents or new registered people. The identifiers of created documents or the information of newly registered people are stored in the HTTP STS by the ADD command in mode ADD_MODE=LAST . The consumers start a little later so that there is data in the queue.

It is also necessary that the consumers do not consume too quickly compared to the producers or it is necessary to manage the case where the list no longer contains a value by detecting that there is no more value available and waiting a few seconds before repeating in a loop.

The consumers consume the values by the commands READ , FIRST , KEEP=FALSE . Here is a schema to explain how the producers ADD and consumers READ in the same queue.

Producer and Consumer With Search By FIND

This is a variation of the previous solution.

Producer

A user with rights limited to a geographical sector creates documents and adds a line with his login + the document number (ex: login23;D12223 ).

login1;D120000
login2;D120210
login23;D12223
login24;D12233
login2;D120214
login23;D12255

Consumer

A user will modify the characteristics of the document, but to do so, he must choose from the list in memory only the lines with the same login as the one he currently uses for questions of rights and geographic sector.

Search in the list with the FIND command and FIND_MODE=SUBSTRING . The string searched for is the login of the connected person (ex: login23) so LINE=login23; (with the separator ; ). In this example, the returned line will be the 1st line login23 and the file number will be used in searching for files in the tested application.

Here, the result of the FIND by substring (SUBSTRING) is: login23;D12223. The line can be consumed with KEEP=FALSE or kept and placed at the end of the list with KEEP=TRUE .

Enrich the Dataset as the Shot Progresses

The idea is to start with a reduced dataset that is read at the start of the shot and to add new lines to the initial dataset as the shot progresses in order to increase or enrich the dataset. The dataset is therefore larger at the end of the shot than at the beginning. At the end of the shot, the enriched dataset can be saved with the SAVE command and the new file can be the future input file for a new shot.

For example, we add people by a scenario. We add to the search dataset these people in the list so the search is done on the first people of the file read at the beginning but also the people added as the performance test progresses.

Verification of the Dataset

The initial dataset can contain values that will generate errors in the script because the navigation falls into a particular case or the entered value is refused because it is incorrect or already exists.

We read the dataset by INITFILE or at the start of the STS, we read (READ) and use the value if the script goes to the end (the Thread Group is configured with " Start Next Thread Loop on Sampler Error"), we save the value by ADD in a file ( valuesok.csv ). At the end of the test, the values of the valuesok.csv file are saved (for example, with a "tearDown Thread Group"). The verified data of the valuesok.csv file will then be used.

Storing the Added Values in a File

In the script, the values added to the application database are saved in a file by the ADD command. At the end of the test, the values in memory are saved in a file (for example, with a "tearDown Thread Group"). The file containing the created values can be used to verify the additions to the database a posteriori or to delete the created values in order to return to an initial state before the test.

A script dedicated to creating a data set can create values and store them in a data file. This file will then be used in the JMeter script during performance tests. On the other hand, a dedicated erase script can take the file of created values to erase them in a loop.

Communication Between Software

The HTTP STS can be used as a means to communicate values between JMeter and other software. The software can be a heavy client, a web application, a shell script, a Selenium test, another JMeter, a LoadRunner, etc.  We can also launch 2 command line tests with JMeter in a row by using an "external standalone" STS as a means to store values created by the first test and used in the second JMeter test. The software can add or read values by calling the URL of the HTTP STS, and JMeter can read these added values or add them itself. This possibility facilitates performance tests but also non-regression tests.

Enrich the Dataset as the Shot Progresses

Gradual Exiting the Test on All JMeter Injectors

In JMeter, there is no notion as in LoadRunner of " GradualExiting ," that is to say, to indicate to the vuser at the end of the iteration if it should continue or not to repeat and therefore stop. We can simulate this " GradualExiting " with the STS and a little code in the JMeter script. With the STS we can load a file " status.csv ," which contains a line with a particular value like the line " RUN ." The vuser asks at the beginning of the iteration the value of the line of the file " status.csv ." If the value is equal to " STOP " then the vuser stops. If the value is zero or different from STOP like " RUN " then the user continues.

Gradual Exiting After Fixed Date Time

We can also program the stop request after a fixed time with this system. We indicate as a parameter the date and time to change the value of the status to STOP ; e.g., 2024-07-31_14h30m45s by a JMeter script that runs in addition to the current load testing. The script is launched, and we calculate the number of milliseconds before the indicated date of the requested stop.

The vuser is put on hold for the calculated duration.

Then the " status.csv " file in the STS is deleted to put the STOP value, which will allow the second JMeter script that is already running to read the status value if status == "STOP" value and to stop properly on all the JMeter injectors or the JMeter alone.

Start the HTTP Simple Table Server

Declaration and Start of the STS by the Graphical Interface

The Simple Table Server is located in the "Non-Test Elements" menu:

Click on the "Start" button to start the HTTP STS.

By default, the directory containing the files is <JMETER_HOME>/bin .

Start From the Command Line

It is possible to start the STS server from the command line.

Or automatically when starting JMeter from the command line (CLI) by declaring the file these 2 lines in jmeter.properties :

jsr223.init.file=simple-table-server.groovy

jmeterPlugin.sts.loadAndRunOnStartup=true

If the value is false , then the STS is not started when launching JMeter from command line without GUI.

The default port is 9191. It can be changed by the property:

# jmeterPlugin.sts.port=9191

If jmeterPlugin.sts.port=0 , then the STS does not start when launching JMeter in CLI mode.

The property jmeterPlugin.sts.addTimestamp=true indicates if the backup of the file (e.g., info.csv) will be prefixed by the date and time (e.g.: 2 0240112T13h00m50s.info.csv ); otherwise, jmeterPlugin.sts.addTimestamp=false writes/overwrites the file info.csv
# jmeterPlugin.sts.addTimestamp=true .

The property jmeterPlugin.sts.daemon=true is used when the STS is launched as an external application with the Linux nohup command (example: nohup ./simple-table-server.sh & ).
In this case, the STS does not listen to the keyboard. Use the <ENTER> key to exit.
The STS enters an infinite loop, so you must call the /sts/STOP command to stop the STS or the killer.

When jmeterPlugin.sts.daemon=false , the STS waits for the entry of <ENTER> to exit. This is the default mode.

Loading Files at Simple Table Server Startup

The STS has the ability to load files into memory at STS startup. Loading files is done when the STS is launched as an external application ( <JMETER_HOME>\bin\simple-table-server.cmd or <JMETER_HOME>/bin/simple-table-server.sh ) and also when JMeter is launched from the command line without GUI or via the JMeter Maven Plugin.

Loading files is not done with JMeter in GUI mode.

The files are read in the directory indicated by the property jmeterPlugin.sts.datasetDirectory , and if this property is null , then in the directory <JMETER_HOME>/bin .

The declaration of the files to be loaded is done by the following properties:

jmeterPlugin.sts.initFileAtStartup=article.csv,filename.csv

jmeterPlugin.sts.initFileAtStartupRegex=false

jmeterPlugin.sts.initFileAtStartup=.+?\.csv

jmeterPlugin.sts.initFileAtStartupRegex=true

When jmeterPlugin.sts.initFileAtStartupRegex=false then the property jmeterPlugin.sts.initFileAtStartup contains the list of files to be loaded with the comma character “ , ” as the file name separator. (e.g., jmeterPlugin.sts.initFileAtStartup=article.csv,filename.csv ). The STS at startup will try to load ( INITFILE ) the files articles.csv then filename.csv.

When jmeterPlugin.sts.initFileAtStartupRegex=true then the property jmeterPlugin.sts.initFileAtStartup contains a regular expression that will be used to match the files in the directory of the property jmeterPlugin.sts.datasetDirectory (e.g.,
jmeterPlugin.sts.initFileAtStartup=.+?\.csv loads into memory ( INITFILE ) all files with the extension " .csv ".

The file name must not contain special characters that would allow changing the reading directory such as ..\..\fichier.csv , /etc/passwd , or ../../../tomcat/conf/server.xml .

The maximum size of a file name is 128 characters (without taking into account the directory path).

Management of the Encoding of Files to Read/Write and the HTML Response

It is possible to define the encoding when reading files or writing data files. The properties are as follows:

Read (INITFILE) or write (SAVE) file with an accent from the text file in the charset like UTF-8 or ISO8859_15

All CSV files need to be in the same charset encoding:

Files will be read ( INITFILE ) with the charset declared by the value of jmeterPlugin.sts.charsetEncodingReadFile .

Files will be written ( SAVE ) with the charset declared by the value of jmeterPlugin.sts.charsetEncodingWriteFile .

The default value jmeterPlugin.sts.charsetEncodingReadFile corresponds to the System property: file.encoding .

The default value jmeterPlugin.sts.charsetEncodingWriteFile corresponds to the System property: file.encoding .

All data files must be in the same charset if they contain non-ASCII characters.

To respond in HTML to different commands, especially READ , the charset found in the response header is indicated by jmeterPlugin.sts.charsetEncodingHttpResponse .

jmeterPlugin.sts.charsetEncodingHttpResponse=<charset> (Use UTF-8) : In the HTTP header add " Content-Type:text/html; charset=<charset> "

The default value is the JMeter property: sampleresult.default.encoding

The list of charsets is declared in the HTML page (take the java.io API column)

For the name of the charset look, see Oracle docs for Supported Encodings .

Column Canonical Name for java.io API and java.lang API

Help With Use

The URL of an STS command is of the form: <HOSTNAME>:<PORT>/sts/<COMMAND>?<PARAMETERS> .

The commands and the names of the parameters are in uppercase (case sensitive).

If no command is indicated then the help message is returned: http://localhost:9191/sts/ .

Commands and Configuration

The following is a list of commands and configuration of the HTTP STS with extracts from the documentation of the JMeter-plugins.org site.

The calls are atomic (with synchronized ) => Reading or adding goes to the end of the current processing before processing the next request.

The commands to the Simple Table Server are performed by HTTP GET and/or POST calls depending on the command.

Documentation is available on the JMeter plugin website .

Distributed Architecture for JMeter

The Simple Table Server runs on the JMeter controller (master) and load generators (slaves) or injectors make calls to the STS to get, find, or add some data.

At the beginning of the test, the first load generator will load data in memory (initial call) and at the end of the test, it asks for the STS saving values in a file.

All the load generators ask for data from the same STS which is started on the JMeter controller.

The INITFILE can also be done at STS startup time (without the first load generator initial call).

Example of a dataset file logins.csv:

The files are read in the directory indicated by the property: jmeterPlugin.sts.datasetDirectory ; if this property is null, then in the directory<JMETER_HOME>/bin/ .

READ: Get One Line From List

http://hostname:port/sts/

READ_MODE=FIRST => login1;password1

READ_MODE=LAST => login5;password5

READ_MODE=RANDOM => login?;password?

KEEP=TRUE => The data is kept and put to the end of the list

KEEP=FALSE => The data is removed

READMULTI: Get Multi Lines From List in One Request

GET Protocol

http://hostname:port/sts/READMULTI?FILENAME=logins.csv&NB_LINES={Nb lines to read}&READ_MODE={FIRST, LAST, RANDOM}&KEEP={TRUE, FALSE}

Available options:

NB_LINES=Number of lines to read : 1 <= Nb lines (Integer) and Nb lines <= list size

READ_MODE=FIRST => Start to read at the first line

READ_MODE=LAST => Start to read at the last line (reverse)

READ_MODE=RANDOM => read n lines randomly

KEEP=TRUE => The data is kept and put to the end of list

KEEP=FALSE => The data is removed

ADD: Add a Line Into a File (GET OR POST HTTP Protocol)

FILENAME=dossier.csv, LINE=D0001123, ADD_MODE={FIRST, LAST}

HTML format:

ADD_MODE=FIRST => Add to the beginning of the list

ADD_MODE=LAST => Add to the end of the list

FILENAME=dossier.csv => If doesn't already exist it creates a LinkList in memory

LINE=1234;98763 =>Tthe line to add

UNIQUE => Do not add a line if the list already contains such a line (return KO)

HTTP POST request:

Method GET:

GET Protocol: http://hostname:port/sts/ADD FILENAME=dossier.csv&LINE=D0001123&ADD_MODE={FIRST, LAST}

FIND: Find a Line in the File (GET OR POST HTTP Protocol)

Command FIND

Find a line ( LINE ) in a file ( FILENAME ) (GET or POST HTTP protocol)

The LINE to find is for FIND_MODE :

A string: SUBSTRING (Default, ALineInTheFile contains the stringToFind ) or EQUALS ( stringToFind == ALineInTheFile )

A regular expression with REGEX_FIND (contains) and REGEX_MATCH (entire region matches the pattern)

KEEP=TRUE => The data is kept and put to the end of the list

KEEP=FALSE => The data is removed

GET Protocol: http://hostname:port/sts/FIND?FILENAME=colors.txt&LINE=(BLUE|RED)&[FIND_MODE=[SUBSTRING,EQUALS,REGEX_FIND,REGEX_MATCH]]&KEEP={TRUE, FALSE}

If find return the first line found, start reading at the first line in the file (linked list):

LENGTH: Return the Number of Remaining Lines of a Linked List

http://hostname:port/sts/LENGTH?FILENAME=logins.csv

HTML format:

STATUS: Display the List of Loaded Files and the Number of Remaining Lines

http://hostname:port/sts/STATUS

HTML format:

SAVE: Save the Specified Linked list in a File to the datasetDirectory Location

http://hostname:port/sts/SAVE?FILENAME=logins.csv

If jmeterPlugin.sts.addTimestamp is set to true, then a timestamp will be added to the filename. The file is stored in jmeterPlugin.sts.datasetDirectory or if null in the <JMETER_HOME>/bin directory: 20240520T16h33m27s.logins.csv.

You can force the addTimestamp value with parameter ADD_TIMESTAMP in the URL like :
http://hostname:port/sts/SAVE?FILENAME=logins.csv &ADD_TIMESTAMP ={true,false}

HTML format:

RESET: Remove All Elements From the Specified List:

http://hostname:port/sts/ RESET ?FILENAME=logins.csv

HTML format:

The Reset command is often used in the “setUp Thread Group” to clear the values in the memory Linked List from a previous test.

It always returns OK even if the file does not exist.

STOP: Shutdown the Simple Table Server

http://hostname:port/sts/ STOP

The stop command is used usually when the HTTP STS server is launched by a script shell and we want to stop the STS at the end of the test.

When the jmeterPlugin.sts.daemon=true , you need to call http://hostname:port/sts/STOP or kill the process to stop the STS.

CONFIG: Display STS Configuration

http://hostname:port/sts/ CONFIG

Display the STS configuration, e.g.:

jmeterPlugin.sts.port=9191 jmeterPlugin.sts.datasetDirectory=null jmeterPlugin.sts.addTimestamp=true jmeterPlugin.sts.demon=false jmeterPlugin.sts.charsetEncodingHttpResponse=UTF-8 jmeterPlugin.sts.charsetEncodingReadFile=UTF-8 jmeterPlugin.sts.charsetEncodingWriteFile=UTF-8 jmeterPlugin.sts.initFileAtStartup= jmeterPlugin.sts.initFileAtStartupRegex=false databaseIsEmpty=false

Error Response KO

When the command and/or a parameter are wrong, the result is a page html status 200 but the title contains the label KO .

Examples:

Send an unknown command. Be careful as the command a case sensitive ( READ != read ).

-DjmeterPlugin.sts.port=<port number>
-DjmeterPlugin.sts.loadAndRunOnStartup=<true/false>
-DjmeterPlugin.sts.datasetDirectory=<path/to/your/directory>
-DjmeterPlugin.sts.addTimestamp=<true/false>
-DjmeterPlugin.sts.daemon=<true/false>
-DjmeterPlugin.sts.charsetEncodingHttpResponse=<charset like UTF-8>
-DjmeterPlugin.sts.charsetEncodingReadFile=<charset like UTF-8>
-DjmeterPlugin.sts.charsetEncodingWriteFile=<charset like UTF-8>
-DjmeterPlugin.sts.initFileAtStartup=<files to read when STS startup, e.g : article.csv,users.csv>
-DjmeterPlugin.sts.initFileAtStartupRegex=false=<false : no regular expression, files with comma separator, true : read files matching the regular expression>

STS in the POM of a Test With the jmeter-maven-plugin

It is possible to use STS in a performance test launched with the jmeter-maven-plugin. To do this:

Put your CSV files in the <project>/src/test/jmeter directory (e.g., logins.csv ).

Put the simple-table-server.groovy (Groovy script) in the <project>/src/test/jmeter directory.

Put your JMeter script in <project>/src/test/jmeter directory (e.g., test_login.jmx ).

Declare in the Maven build section, in the configuration <jmeterExtensions> declare the artifact kg.apc:jmeter-plugins-table-server:<version> .

Declare user properties for STS configuration and automatic start.

If you use a localhost and a proxy configuration, you could add a proxy configuration with <hostExclusions>localhost</hostExclusions> .

Extract pom.xml dedicated to HTTP Simple Table Server :

<groupId>com.lazerycode.jmeter</groupId> <artifactId>jmeter-maven-plugin</artifactId> <version>3.8.0</version> <configuration> <jmeterExtensions> <artifact>kg.apc:jmeter-plugins-table-server:5.0</artifact> </jmeterExtensions> <propertiesUser> <!-- properties configuration for Http Simple Table Server with automatic start when JMeter start --> <jmeterPlugin.sts.port>9191</jmeterPlugin.sts.port> <jmeterPlugin.sts.addTimestamp>true</jmeterPlugin.sts.addTimestamp> <jmeterPlugin.sts.datasetDirectory>${project.build.directory}/jmeter/testFiles</jmeterPlugin.sts.datasetDirectory> <jmeterPlugin.sts.loadAndRunOnStartup>true</jmeterPlugin.sts.loadAndRunOnStartup> <jsr223.init.file>${project.build.directory}/jmeter/testFiles/simple-table-server.groovy</jsr223.init.file> </propertiesUser> </configuration> </plugin> </plugins> </build>