ThingsBoard supports user-defined functions (UDF) for data processing in the Rule Engine and Data Converters . The original programming language for the UDF is JavaScript. It is popular, well-known, and simple. We plan to support JavaScript forever. Nevertheless, we have decided to provide an alternative to JavaScript. You may find our motivation below.

Motivation

ThingsBoard is written in Java and currently uses Java 11. There are two ways to execute the JS function in ThingsBoard:

A) use remote JS Executor microservice written in Node.js. It is the default way for ThingsBoard to run in a cluster/microservices mode;

B) use local JS Executor powered by Nashorn JS engine that runs inside the JVM. It is the default way for ThingsBoard while deployed in monolithic mode.

The Nashorn JS engine is now deprecated and is removed from Java 16. It is also relatively slow. Some users already called Nashorn to be the platform’s kryptonite. That is why it was absolutely clear that we need a replacement to Nashorn. Many users suggested GraalVM for its built-in polyglot feature, but it was not suitable for us for multiple reasons. The most important reason is security and the ability to control every aspect of the UDF execution. Besides, most UDFs are relatively simple functions that transform or filter data, and we want to find a more effective way to execute them.

Our search for existing Script/Expression Language (EL) implementations lead us to the MVEL . The ThingsBoard Expression Language (TBEL) is basically a fork of MVEL with some important security constraints, built-in memory management, and frequently used helper functions that are specific to ThingsBoard.

TBEL vs Nashorn

TBEL is lightweight and is super fast comparing to Nashorn. For example, the execution of 1000 simple scripts like: “return msg.temperature > 20” took 16 seconds for Nashorn and 12 milliseconds for MVEL. More than 1000 times faster. We have forked the TBEL codebase and added additional security improvements to ensure no CPU or memory abuse. So, no need to run Nashorn with the sandbox environment.

Of course, TBEL is not as powerful as JS, but most of the use cases do not need this. The one who requires JS flexibility may use remote JS Executors as usual.

TBEL vs JS Executors

JS Executors is a separate microservice based on Node.js. It is powerful and supports the latest JS language standards. However, there is a noticeable overhead to execute JS functions remotely. The Rule Engine and JS executors communicate through the queue. This process consumes resources and introduces a relatively small latency (a few ms). The latter may become a problem if you have multiple rule nodes chained into one rule chain.

TBEL execution consumes much less resources and has no extra latency for inter process communications.

TBEL Language Guide

TBEL is used to evaluate expressions written using Java syntax. Unlike Java however, TBEL is dynamically typed (with optional typing), meaning type qualification is not required in the source. The TBEL expression can be as simple as a single identifier, or as complicated as an expression with method calls and inline collections.

Simple Property Expression

msg.temperature
In this expression, we simply have a single identifier (msg.temperature), which by itself, is what we refer to in TBEL as a property expression, 
in that the only purpose of the expression is to extract a property out of a variable or context object.
TBEL can even be used for evaluating a boolean expression. 
Assuming you are using TBEL in the Rule Engine to define simple script filter node:

return msg.temperature > 10;
Like Java, TBEL supports the full gamut of operator precedence rules, including the ability to use bracketing to control execution order.

return (msg.temperature > 10 && msg.temperature < 20) || (msg.humidity > 10 && msg.humidity < 60);
Multiple statements
You may write scripts with an arbitrary number of statements using the semi-colon to denote the termination of a statement. 
This is required in all cases except in cases where there is only one statement, or for the last statement in a script.

var a = 2;
var b = 2;
return a + b
Note the lack of semi-colon after ‘a + b’. New lines are not substitutes for the use of the semi-colon in MVEL.
Value Coercion
MVEL’s type coercion system is applied in cases where two incomparable types are presented by attempting to coerce the ‘’right’‘value to that of the type of the’‘left’’ value, and then vice-versa.
For example:

"123" == 123;
This expression is true in TBEL because the type coercion system will coerce the untyped number 123 to a String in order to perform the comparison.
TBEL allows you to create Maps. We use our own implementation of the Map to control memory usage. 
That is why, TBEL allows inline creation of the maps only. Most common operation with the map:

// Create new map
var map = { "temperature": 42, "nested" : {"rssi": 130}};
// Change value of the key
map.temperature = 0;
// Add new key
map.humidity = 73;
// Check existance of the key
if(map.temperature != null){
// Null-Safe expressions using ?
if(map.?nonExistingKey.smth > 10){
// Iterate through the map
foreach(element : map.entrySet()){
    // Get the key
    element.key
    // Get the value
    element.value;
// remove value from the map
map.remove("temperature");
// get map size
map.size();
Lists
TBEL allows you to create Lists. We use our own implementation of the List to control memory usage of the script. 
That is why, TBEL allows inline creation of the lists only. For example:

// Create new list
var list = ["A", "B", "C"];
// List elelement access
list[0];
// Add element
list.add("F");
// Add element using index
list.add(3, "D");
// Remove element by index
list.remove(4);
// Remove element by value
list.remove("D");
// Set element using index
list[2] = "C";
list.set(2, "C");
// Size of the list
list.size();
// Get sub list - JS style
list.slice(1, 3);
// Foreach 
for (item: list) { 
    var smth = item;
// For loop 
for (int i =0; i < list.size; i++) { 
    var smth = list[i];
Arrays
TBEL allows you to create Arrays. To control the memory usage, we allow arrays of primitive types only. String arrays are automatically converted to lists.

// Create new array
int[] array = new int[3];
array[0] = 1;
array[1] = 2;
array[2] = 3;
str = "My String";
str[0]; // returns 'M';
function sum(list){
    int result = 0;
    for(var i = 0; i < list.length; i++){
        result += list[i];
    return result;
sum(array); // returns 6
array[3] = 4; // Will cause ArrayIndexOutOfBoundsException
Literals
A literal is used to represent a fixed-value in the source of a particular script.
String literals
String literals may be denoted by single or double quotes.

"This is a string literal"
'This is also string literal'
String escape sequences:
  
\ - Double escape allows rendering of single backslash in string.
  
\n - Newline
  
\r - Return
  
\u#### - Unicode character (Example: \uAE00)
  
### - Octal character (Example: \73)
Numeric literals
Integers can be represented in decimal (base 10), octal (base 8), or hexadecimal (base 16).
A decimal integer can be expressed as any number that does not start with zero.

125 // decimal
An octal representation of an integer is possible by prefixing the number with a zero, followed by digits ranging from 0 to 7.

0353 // octal
Hexidecimal is represented by prefixing the integer with 0x followed by numbers ranging from 0-9..A-F.

0xAFF0 // hex
A floating point number consists of a whole number and a factional part denoted by the point/period character, with an optional type suffix.

10.503 // a double
94.92d // a double
14.5f // a float
You can represent *BigInteger* and *BigDecimal* literals by using the suffixes B and I (uppercase is mandatory).

104.39484B // BigDecimal
8.4I // BigInteger
Boolean literals are represented by the reserved keywords true and false.
The null literal is denoted by the reserved keywords null or nil.
Using Java Classes
The TBEL implementation allows usage of some Java classes from the java.util and java.lang packages. For example:

var foo = java.lang.Math.sqrt(4);
For the security purpose, usage of those classes is constrained. You are able to call both static and non-static methods, but you are not able to assign the instance of the class to the variable:




    

var list = ["A", "B", "C"];
java.util.Collections.reverse(list); // allowed
list = new java.util.ArrayList(); // Not allowed
To simplify migration from the JS, we have added the JSON class with static methods: JSON.stringify and JSON.parse that work similar to JS. For example:
For the same purpose, we have added Date class that you are able to use without the package name.
Flow Control
If-Then-Else
TBEL supports full, C/Java-style if-then-else blocks. For example:

if (temperature > 0) {
   return "Greater than zero!";
} else if (temperature == -1) { 
   return "Minus one!";
} else { 
   return "Something else!";
Ternary statements
Ternary statements are supported just as in Java:

temperature > 0 ? "Yes" : "No";
Nested ternary statements are also supported.
Foreach
One of the most powerful features in TBEL is it’s foreach operator. It is similar to the for each operator in Java 1.5 in both syntax and functionality. 
It accepts two parameters separated by a colon, the first is the local variable for the current element, and the second is the collection or array to be iterated.
For example:

sum = 0;
for (n : numbers) {
   sum+=n;
Since TBEL treats Strings as iterable objects you can iterate a String (character by character) with a foreach block:

str = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
for (c : str) {
   //do something 
For Loop

var sum = 0;
for (int i =0; i < 100; i++) { 
   sum += i;
Do While, Do Until
do while and do until are implemented in TBEL, following the same convention as Java, with until being the inverse of while.

do { 
   x = something();
while (x != null);
… is semantically equivalent to …

do {
   x = something();
until (x == null);
While, Until
TBEL implements standard while, with the addition of the inverse until.

while (isTrue()) {
   doSomething();
… or …

until (isFalse()) {
   doSomething();
Helper functions
Creates a Base64-encoded ASCII string from a binary string (i.e., a string in which each character in the string is treated as a byte of binary data)
Syntax:
String btoa(String input)
Parameters:
  
input: string - The binary string to encode.
Return value:
An ASCII string containing the Base64 representation of the input.
Examples:

var encodedData = btoa("Hello, world"); // encode a string
var decodedData = atob(encodedData); // decode the string
Decodes a string of data which has been encoded using Base64 encoding.
Syntax:
String atob(String input)
Parameters:
  
input: string - A binary string containing base64-encoded data.
Return value:
An ASCII string containing decoded data from encodedData.
Examples:

var encodedData = btoa("Hello, world"); // encode a string
var decodedData = atob(encodedData); // decode the string
bytesToString
Creates a string from the list of bytes
Syntax:
*String bytesToString(List   bytesList[, String charsetName])*
Parameters:
  
bytesList: List of Bytes - A list of bytes.
  
charsetName: String - optional Charset name. UTF_8 by default.
Return value:
A string constructed from the specified byte list.
Examples:

var bytes = [(byte)0x48,(byte)0x45,(byte)0x4C,(byte)0x4C,(byte)0x4F];
return bytesToString(bytes); // Returns "HELLO"
decodeToString
Alias for the bytesToString
decodeToJson
Decodes a list of bytes to the JSON document.
Syntax:
*String decodeToJson(List   bytesList)*
Parameters:
  
bytesList: List of Bytes - A list of bytes.
Return value:
A JSON object or primitive.
Examples:

var base64Str = "eyJoZWxsbyI6ICJ3b3JsZCJ9"; // Base 64 representation of the '{"hello": "world"}'
var bytesStr = atob(base64Str);
var bytes = stringToBytes(bytesStr);
return decodeToJson(bytes); // Returns '{"hello": "world"}'
stringToBytes
Converts input binary string to the list of bytes.
Syntax:
*List   stringToBytes(String input[, String charsetName])*
Parameters:
  
input: Binary string - string in which each character in the string is treated as a byte of binary data.
  
charsetName: String - optional Charset name. UTF_8 by default.
Return value:
A list of bytes.
Examples:

var base64Str = "eyJoZWxsbyI6ICJ3b3JsZCJ9"; // Base 64 representation of the '{"hello": "world"}' 
var bytesStr = atob(base64Str);
return stringToBytes(bytesStr); // Returns [123, 34, 104, 101, 108, 108, 111, 34, 58, 32, 34, 119, 111, 114, 108, 100, 34, 125]
parseInt
Converts input string to integer.
Syntax:
Integer parseInt(String str[, String radix])
Parameters:
  
str: string - the String containing the integer representation to be parsed.
  
radix: String - optional radix to be used while parsing string.
Return value:
An integer value.
Examples:

return parseInt("0") // returns 0
return parseInt("473") // returns 473
return parseInt("+42") // returns 42
return parseInt("-0", 10) // returns 0
return parseInt("-0xFF") // returns -255        
return parseInt("-FF", 16) // returns -255
return parseInt("1100110", 2) // returns 102
return parseInt("2147483647", 10) // returns 2147483647
return parseInt("-2147483648", 10) // returns -2147483648
return parseInt("2147483648", 10) throws a NumberFormatException
return parseInt("99", 8) throws a NumberFormatException
return parseInt("Kona", 10) throws a NumberFormatException
return parseInt("Kona", 27) // returns 411787
parseFloat
Converts input string to float.
Syntax:
Integer parseFloat(String str)
Parameters:
  
str: string - the string to be parsed.
Return value:
A float value.
Examples:

return parseFloat("4.2"); // returns 4.2
parseDouble
Converts input string to double.
Syntax:
Integer parseDouble(String str)
Parameters:
  
str: string - the string to be parsed.
Return value:
A double precision value.
Examples:

return parseDouble("4.2"); // returns 4.2
parseHexToInt
Converts the hex string to integer.
Syntax:
int parseHexToInt(String hex[, boolean bigEndian])
Parameters:
  
hex: string - the hex string with big-endian byte order.
  
bigEndian: boolean - the big-endian (BE) byte order if true, little-endian (LE) otherwise.
Return value:
Parsed integer value.
Examples:

return parseHexToInt("BBAA"); // returns 48042
return parseHexToInt("BBAA", true); // returns 48042
return parseHexToInt("AABB", false); // returns 48042
return parseHexToInt("BBAA", false); // returns 43707
parseLittleEndianHexToInt
Alias for parseHexToInt(hex, false)




    

Syntax:
int parseLittleEndianHexToInt(String hex)
parseBigEndianHexToInt
Alias for parseHexToInt(hex, true)
Syntax:
int parseBigEndianHexToInt(String hex)
toFixed
Rounds the double value towards “nearest neighbor”.
Syntax:
double toFixed(double value, int precision)
Parameters:
  
value: double - the double value.
  
precision: int - the precision.
Return value:
Rounded double
Examples:

return toFixed(0.345, 1); // returns 0.3
return toFixed(0.345, 2); // returns 0.35
hexToBytes
Converts the hex string to list of integer values, where each integer represents single byte.
Syntax:
*List   hexToBytes(String hex)*
Parameters:
  
hex: string - the hex string with big-endian byte order.
Return value:
Parsed list of integer values.
Examples:

return hexToBytes("BBAA"); // returns [187, 170]
bytesToHex
Converts the list of integer values, where each integer represents single byte, to the hex string.
Syntax:
*String bytesToHex(List   bytes)*
Parameters:
  
bytes: List of integer - the list of integer values, where each integer represents single byte.
Return value:
Hex string.
Examples:

return bytesToHex([187, 170]); // returns "BBAA"
bytesToBase64
Converts the byte array, to Base64 string.
Syntax:
String bytesToBase64(byte[] bytes)
Parameters:
  
bytes: List of integer - the list of integer values, where each integer represents single byte.
Return value:
Base64 string.
Examples:

return bytesToBase64([42, 73]); // returns "Kkk="
base64ToHex
Decodes the Base64 string, to hex string.
Syntax:
String base64ToHex(String input)
Parameters:
  
input: String - the Base64 string.
Return value:
Hex string
Examples:

return base64ToHex("Kkk="); // returns "2A49"
base64ToBytes
Decodes the Base64 string, to byte array.
Syntax:
byte[] base64ToBytes(String input)
Parameters:
  
input: String - the Base64 string.
Return value:
Byte array.
Examples:

return base64ToBytes("Kkk="); // returns [42, 73]
parseBytesToInt
Parses int from the byte array given the offset, length and optional endianness.
Syntax:
*int parseBytesToInt([byte[] or List  ] data, int offset, int length[, boolean bigEndian])*
Parameters:
  
data: byte[] or List of Byte - the byte array.
  
offset: int - the offset in the array.
  
length: int - the length in bytes. Less then or equal to 4.
  
bigEndian: boolean - optional, LittleEndian if false, BigEndian otherwise.
Return value:
integer value.
Examples:

return parseBytesToInt(new byte[]{(byte) 0xAA, (byte) 0xBB, (byte) 0xCC, (byte) 0xDD}, 0, 3, true); // returns 11189196 in Decimal or 0xAABBCC  
return parseBytesToInt(new byte[]{(byte) 0xAA, (byte) 0xBB, (byte) 0xCC, (byte) 0xDD}, 0, 3, false); // returns 13417386 in Decimal or 0xCCBBAA
toFlatMap
Iterates recursive over the given object and creates a single level json object.

If the incoming message contains arrays, key for transformed value will contain index of the element.
Syntax:

Map<String, Object> toFlatMap(Map<String, Object> json)
Map<String, Object> toFlatMap(Map<String, Object> json, boolean pathInKey)
Map<String, Object> toFlatMap(Map<String, Object> json, List<String> excludeList)
Map<String, Object> toFlatMap(Map<String, Object> json, List<String> excludeList, boolean pathInKey)
Parameters:
  
json Map<String, Object> - input JSON object.
  
excludeList List<String> - optional List with keys, for exclude from result. Default: [].
  
pathInKey boolean - optional Add path to keys. Default: true.
    "key_to_overwrite": "root_value",
    "key11": {
        "key_to_overwrite": "second_level_value"
toFlatMap(json)
Expected behavior - get the single level object with paths in keys.

return toFlatMap(json);
Output:
    "key1": "value1",
    "key2": 12,
    "key3.key4": "value4",
    "key3.key5": 34,
    "key3.key6.0.key7": "value7",
    "key3.key6.1.key8": "value8",
    "key3.key6.2": "just_string_value_in_array",
    "key3.key6.3": 56,
    "key3.key9.key10.0": "value10_1",
    "key3.key9.key10.1": "value10_2",
    "key3.key9.key10.2": "value10_3",
    "key_to_overwrite": "root_value",
    "key11.key_to_overwrite": "second_level_value"
toFlatMap(json, pathInKey)
Expected behavior - get the single level object without paths in keys.

key_to_overwrite should contain the value from key11.key_to_overwrite.

return toFlatMap(json, false);
Output:
    "key1": "value1",
    "key2": 12,
    "key4": "value4",
    "key5": 34,
    "key7": "value7",
    "key8": "value8",
    "key6.2": "just_string_value_in_array",
    "key6.3": 56,
    "key10.0": "value10_1",
    "key10.1": "value10_2",
    "key10.2": "value10_3",
    "key_to_overwrite": "second_level_value"
As you can see, key_to_overwrite is second_level_value instead of root_value.
toFlatMap(json, excludeList)
Expected behavior - get the single level object with paths in keys, without keys key1 and key3.

return toFlatMap(json, ["key1", "key3"]);
Output:
    "key2": 12,
    "key_to_overwrite": "root_value",
    "key11.key_to_overwrite": "second_level_value"
As you can see, key1 and key3 was removed from output. Excluding works on any level with incoming keys.
toFlatMap(json, excludeList, pathInKey)
Expected behavior - get the single level object without paths in keys, without keys key2 and key4.

key_to_overwrite should contain the value from key11.key_to_overwrite.

return toFlatMap(json, ["key2", "key4"], false);
Output:
    "key1": "value1",
    "key5": 34,
    "key7": "value7",
    "key8": "value8",
    "key6.2": "just_string_value_in_array",
    "key6.3": 56,
    "key10.0": "value10_1",
    "key10.1": "value10_2",
    "key10.2": "value10_3",
    "key_to_overwrite": "second_level_value"
As you can see, key2 and key4 was excluded from the output and key_to_overwrite is second_level_value.
        Cookie policy
        We use cookies to improve user experience. By continuing to browse this site, you agree the use of cookies, in accordance with our cookie policy.
    Accept