● rabbitmq-server.service - RabbitMQ broker
Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabl>
Active: active (running) since Tue 2023-03-28 17:16:40 CST; 16h ago
Main PID: 196280 (beam.smp)
Tasks: 25 (limit: 9507)
Memory: 73.8M
CPU: 4min 27.888s
CGroup: /system.slice/rabbitmq-server.service
├─196280 /usr/lib/erlang/erts-13.2/bin/beam.smp -W w -MBas ageffcbf -MHas ageffcbf>
├─196290 erl_child_setup 32768
├─197086 /usr/lib/erlang/erts-13.2/bin/epmd -daemon
├─197156 /usr/lib/erlang/erts-13.2/bin/inet_gethost 4
├─197157 /usr/lib/erlang/erts-13.2/bin/inet_gethost 4
└─197813 /bin/sh -s rabbit_disk_monitor
TB_QUEUE_TYPE=rabbitmq, DATABASE=postgres, MONITORING_ENABLED=true
queue-rabbitmq.env
TB_QUEUE_RABBIT_MQ_HOST=192.168.1.206
TB_QUEUE_RABBIT_MQ_PORT=5672
TB_QUEUE_RABBIT_MQ_USERNAME=guest
TB_QUEUE_RABBIT_MQ_PASSWORD=guest
network state
tcp 0 0 127.0.0.1:5672 0.0.0.0:* LISTEN 196280/beam .smp
tcp 0 0 0.0.0.0:25672 0.0.0.0:* LISTEN 196280/beam .smp
tcp6 0 0 ::1:5672 :::* LISTEN 196280/beam .smp
docker-compose logs -f
$ docker-compose logs -f | grep ERROR
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
docker-tb-js-executor-5 | 2023-03-28 09:23:52,226 [main] ERROR: Failed to start ThingsBoard JavaScript Executor Microservice: connect ECONNREFUSED 192.168.1.206:5672
docker-tb-js-executor-5 | 2023-03-28 09:23:52,227 [main] ERROR: Error: connect ECONNREFUSED 192.168.1.206:5672
docker-tb-vc-executor1-1 | 2023-03-28 09:25:31,454 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-vc-executor1-1 | 2023-03-28 09:25:32,320 [main] ERROR o.s.boot.SpringApplication - Application run failed
docker-tb-vc-executor1-1 | 2023-03-28 09:26:26,679 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-core2-1 | 2023-03-28 09:28:11,195 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-mqtt-transport2-1 | 2023-03-28 09:25:52,227 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-mqtt-transport2-1 | 2023-03-28 09:25:53,135 [main] ERROR o.s.boot.SpringApplication - Application run failed
docker-tb-core2-1 | 2023-03-28 09:28:11,230 [main] ERROR o.s.b.w.e.tomcat.TomcatStarter - Error starting Tomcat context. Exception: org.springframework.beans.factory.UnsatisfiedDependencyException. Message: Error creating bean with name 'thingsboardSecurityConfiguration': Unsatisfied dependency expressed through field 'oauth2AuthenticationSuccessHandler'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'oauth2AuthenticationSuccessHandler' defined in URL [jar:file:/usr/share/thingsboard/bin/thingsboard.jar!/BOOT-INF/classes!/org/thingsboard/server/service/security/auth/oauth2/Oauth2AuthenticationSuccessHandler.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'OAuth2ClientMapperProvider': Unsatisfied dependency expressed through field 'basicOAuth2ClientMapper'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'basicOAuth2ClientMapper': Unsatisfied dependency expressed through field 'tbUserService'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'defaultUserService' defined in URL [jar:file:/usr/share/thingsboard/bin/thingsboard.jar!/BOOT-INF/classes!/org/thingsboard/server/service/entitiy/user/DefaultUserService.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'defaultMailService' defined in URL [jar:file:/usr/share/thingsboard/bin/thingsboard.jar!/BOOT-INF/classes!/org/thingsboard/server/service/mail/DefaultMailService.class]: Unsatisfied dependency expressed through constructor parameter 3; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'defaultTbApiUsageReportClient' defined in URL [jar:file:/usr/share/thingsboard/bin/thingsboard.jar!/BOOT-INF/lib/queue-3.4.4.jar!/org/thingsboard/server/queue/usagestats/DefaultTbApiUsageReportClient.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'tbCoreQueueProducerProvider' defined in URL [jar:file:/usr/share/thingsboard/bin/thingsboard.jar!/BOOT-INF/lib/queue-3.4.4.jar!/org/thingsboard/server/queue/provider/TbCoreQueueProducerProvider.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'rabbitMqTbCoreQueueFactory' defined in URL [jar:file:/usr/share/thingsboard/bin/thingsboard.jar!/BOOT-INF/lib/queue-3.4.4.jar!/org/thingsboard/server/queue/provider/RabbitMqTbCoreQueueFactory.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.thingsboard.server.queue.provider.RabbitMqTbCoreQueueFactory]: Constructor threw exception; nested exception is java.lang.RuntimeException: Failed to create connection.
docker-tb-core2-1 | 2023-03-28 09:28:13,318 [main] ERROR o.s.boot.SpringApplication - Application run failed
docker-tb-rule-engine1-1 | 2023-03-28 09:27:55,228 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-rule-engine1-1 | 2023-03-28 09:27:56,379 [main] ERROR o.s.boot.SpringApplication - Application run failed
docker-tb-rule-engine1-1 | 2023-03-28 09:31:33,203 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-rule-engine1-1 | 2023-03-28 09:31:35,334 [main] ERROR o.s.boot.SpringApplication - Application run failed
docker-tb-js-executor-6 | 2023-03-28 09:23:53,234 [main] ERROR: Failed to start ThingsBoard JavaScript Executor Microservice: connect ECONNREFUSED 192.168.1.206:5672
docker-tb-js-executor-6 | 2023-03-28 09:23:53,234 [main] ERROR: Error: connect ECONNREFUSED 192.168.1.206:5672
docker-tb-lwm2m-transport-1 | 2023-03-28 09:25:57,869 [main] ERROR o.t.s.queue.rabbitmq.TbRabbitMqAdmin - Failed to create connection.
docker-tb-lwm2m-transport-1 | 2023-03-28 09:25:58,751 [main] ERROR o.s.boot.SpringApplication - Application run failed
docker-tb-js-executor-7 | 2023-03-28 09:23:53,951 [main] ERROR: Failed to start ThingsBoard JavaScript Executor Microservice: connect ECONNREFUSED 192.168.1.206:5672
one more finding - tb-js-executor restarting from time to time
$ docker-compose -f docker-compose.yml ps
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "JAVA_OPTS" variable is not set. Defaulting to a blank string.
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
docker-grafana-1 grafana/grafana "/run.sh" grafana 2 days ago Up 59 minutes 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp
docker-postgres-1 postgres:12 "docker-entrypoint.s…" postgres 2 days ago Up About an hour 0.0.0.0:54077->5432/tcp, :::54074->5432/tcp
docker-prometheus-1 prom/prometheus:v2.1.0 "/bin/prometheus --c…" prometheus 2 days ago Up 59 minutes 0.0.0.0:9090->9090/tcp, :::9090->9090/tcp
docker-redis-1 bitnami/redis:7.0 "/opt/bitnami/script…" redis 2 days ago Up About an hour 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp
docker-tb-coap-transport-1 thingsboard/tb-coap-transport:latest "start-tb-coap-trans…" tb-coap-transport 59 minutes ago Up 51 seconds 0.0.0.0:5683->5683/udp, :::5683->5683/udp
docker-tb-core1-1 thingsboard/tb-node:latest "start-tb-node.sh" tb-core1 59 minutes ago Up 2 minutes 0.0.0.0:54533->7070/tcp, :::54530->7070/tcp, 0.0.0.0:54532->8080/tcp, :::54529->8080/tcp
docker-tb-core2-1 thingsboard/tb-node:latest "start-tb-node.sh" tb-core2 59 minutes ago Up 2 minutes 0.0.0.0:54530->7070/tcp, :::54527->7070/tcp, 0.0.0.0:54529->8080/tcp, :::54526->8080/tcp
docker-tb-http-transport1-1 thingsboard/tb-http-transport:latest "start-tb-http-trans…" tb-http-transport1 59 minutes ago Up 41 seconds 0.0.0.0:54545->8081/tcp, :::54542->8081/tcp
docker-tb-http-transport2-1 thingsboard/tb-http-transport:latest "start-tb-http-trans…" tb-http-transport2 59 minutes ago Up 49 seconds 0.0.0.0:54543->8081/tcp, :::54540->8081/tcp
docker-tb-js-executor-1 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 32 seconds ago
docker-tb-js-executor-10 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 9 seconds ago
docker-tb-js-executor-2 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 3 seconds ago
docker-tb-js-executor-3 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 11 seconds ago
docker-tb-js-executor-4 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 3 seconds ago
docker-tb-js-executor-5 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Up 4 seconds
docker-tb-js-executor-6 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 56 seconds ago
docker-tb-js-executor-7 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 37 seconds ago
docker-tb-js-executor-8 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 2 seconds ago
docker-tb-js-executor-9 thingsboard/tb-js-executor:latest "docker-entrypoint.s…" tb-js-executor 59 minutes ago Restarting (255) 35 seconds ago
docker-tb-lwm2m-transport-1 thingsboard/tb-lwm2m-transport:latest "start-tb-lwm2m-tran…" tb-lwm2m-transport 59 minutes ago Up 28 seconds 0.0.0.0:5685->5685/udp, :::5685->5685/udp
docker-tb-mqtt-transport1-1 thingsboard/tb-mqtt-transport:latest "start-tb-mqtt-trans…" tb-mqtt-transport1 59 minutes ago Up 47 seconds 0.0.0.0:54544->1883/tcp, :::54541->1883/tcp
docker-tb-mqtt-transport2-1 thingsboard/tb-mqtt-transport:latest "start-tb-mqtt-trans…" tb-mqtt-transport2 59 minutes ago Up 32 seconds 0.0.0.0:54546->1883/tcp, :::54543->1883/tcp
docker-tb-rule-engine1-1 thingsboard/tb-node:latest "start-tb-node.sh" tb-rule-engine1 59 minutes ago Up About a minute 0.0.0.0:54540->8080/tcp, :::54537->8080/tcp
docker-tb-rule-engine2-1 thingsboard/tb-node:latest "start-tb-node.sh" tb-rule-engine2 59 minutes ago Up About a minute 0.0.0.0:54539->8080/tcp, :::54536->8080/tcp
docker-tb-snmp-transport-1 thingsboard/tb-snmp-transport:latest "start-tb-snmp-trans…" tb-snmp-transport 59 minutes ago Up 57 seconds
docker-tb-vc-executor1-1 thingsboard/tb-vc-executor:latest "start-tb-vc-executo…" tb-vc-executor1 59 minutes ago Up 5 seconds 0.0.0.0:54548->8081/tcp, :::54545->8081/tcp
docker-tb-vc-executor2-1 thingsboard/tb-vc-executor:latest "start-tb-vc-executo…" tb-vc-executor2 59 minutes ago Up 25 seconds 0.0.0.0:54547->8081/tcp, :::54544->8081/tcp
docker-tb-web-ui1-1 thingsboard/tb-web-ui:latest "docker-entrypoint.s…" tb-web-ui1 2 days ago Up About an hour 0.0.0.0:54078->8080/tcp, :::54075->8080/tcp
docker-tb-web-ui2-1 thingsboard/tb-web-ui:latest "docker-entrypoint.s…" tb-web-ui2 2 days ago Up About an hour 0.0.0.0:54079->8080/tcp, :::54076->8080/tcp
docker-zookeeper-1 zookeeper:3.8.0 "/docker-entrypoint.…" zookeeper 2 days ago Up About an hour 2888/tcp, 3888/tcp, 8080/tcp, 0.0.0.0:54080->2181/tcp, :::54077->2181/tcp
haproxy-certbot thingsboard/haproxy-certbot:1.3.0 "/start.sh" haproxy 59 minutes ago Up 58 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:1883->1883/tcp, :::1883->1883/tcp, 0.0.0.0:7070->7070/tcp, :::7070->7070/tcp, 0.0.0.0:9999->9999/tcp, :::9999->9999/tcp
ECONNREFUSED - Connection refused by the server, so you need to check your credentials.
Also, please check your firewall and ports that you listen.
Maybe you cannot access your RabbitMQ from the docker network or vice versa.
In addition, your queue cannot start and also ThingsBoard cannot start too.
Regards,
Denys
@den1jkee thanks!
Some new findings update, all tests were based on the same virtual machine (Debian 11.6)
release-3.2 cluster setup with Docker Compose, .env (TB_QUEUE_TYPE=kafka, TB_VERSION=3.2.2), docker-compose.yml (js-executor scala from 20 to 2) --- Just wait a long moment and we can login via tb IP address (http://192.168.1.xxx) successfully.
However, the VM might run out of memory to become so slow. So we see 503 Service Unavailable from Web UI again.
release-3.4 cluster setup with Docker Compose, (TB_QUEUE_TYPE=kafka, TB_VERSION=latest) --- Just wait a long long moment and we can login via tb IP address (http://192.168.1.xxx) successfully.
For TB_QUEUE_TYPE=rabbitmq, guest/guest is not allowed. We need to new create admin account for queue-rabbitmq.env and exact IP address instead of localhost.
For lots of microservices (docker images running), how do we allocate proper memory** via JAVA_OPTS to ensure cluster mode can be stable running as monolithic architecture if we allocate 8G memory to one VM or we run 2 VMs with 4G memory for each?
Or a cost effective HA based on microservice architecture?
You can always set Java XMS, XMX variable for restriction of consuming RAM and if you using microservices you can adjust scale variable for a count of JS Executors, by default it is set 10 if I remember correctly.
These variables can be helpful for instances with low RAM amounts.
Best regards,
Denys
@den1jkee thanks!
Some new findings update, all tests were based on the same virtual machine (Debian 11.6)
1. release-3.2 cluster setup with Docker Compose, .env (TB_QUEUE_TYPE=kafka, TB_VERSION=3.2.2), docker-compose.yml (js-executor scala from 20 to 2) --- Just wait a long moment and we can login via tb IP address (http://192.168.1.xxx) successfully.
However, the VM might run out of memory to become so slow. So we see 503 Service Unavailable from Web UI again.
2. release-3.4 cluster setup with Docker Compose, (TB_QUEUE_TYPE=kafka, TB_VERSION=latest) --- Just wait a long long moment and we can login via tb IP address (http://192.168.1.xxx) successfully.
3. For TB_QUEUE_TYPE=rabbitmq, guest/guest is not allowed. We need to new create admin account for queue-rabbitmq.env and exact IP address instead of localhost.
For lots of microservices (docker images running), how do we allocate proper memory** via JAVA_OPTS to ensure cluster mode can be stable running as monolithic architecture if we allocate 8G memory to one VM or we run 2 VMs with 4G memory for each?
Or a cost effective HA based on microservice architecture?
Hi, I'm experiencing the same behaviour.
Tried with TB_VERSION=3.4.4 & TB_VERSION=latest, but after all services are up, I'm unable to login due 503 Service Unavailable
The question is: what are the minimum hardware requirements to deploy the latest version (cluster with docker-compose)? As after the services have been up for some time, it makes impossible to login as VM run out of memory and becomes almost unresponsive. So we see 503 Service Unavailable from Web UI.
Update:
We've checked that after waiting ~20' since executing the command ./docker-start-services.sh we are finally able to Login into the platform and the message 503 Service Unavailable is no longer displayed.
VM resourcers (CPU/RAM) stucks at 100% for a while since services start, but afterwards resources are freed up.
So, @star7128, my advise is to wait for a while and check if it works for you as well.
Please, keep me updated ;-)
Hi @alexcorvis84,
How many resources does your VM have and how many JS Executors are you running?
Best regards,
Denys
Hi @alexcorvis84, How many resources does your VM have and how many JS Executors are you running?
Best regards, Denys
Hi @den1jkee,
Now I'm running Thingsboard version 3.5 with just 5 JS Executors:
Name Command State Ports
--------------------------------------------------------------------------------------------------------------------------------------------------------------
docker_redis_1 docker-entrypoint.sh redis ... Up 0.0.0.0:49879->6379/tcp,:::49879->6379/tcp
docker_tb-coap-transport_1 start-tb-coap-transport.sh Up 0.0.0.0:5683->5683/udp,:::5683->5683/udp
docker_tb-core1_1 start-tb-node.sh Up 0.0.0.0:49890->7070/tcp,:::49890->7070/tcp, 0.0.0.0:49889->8080/tcp,:::49889->8080/tcp
docker_tb-core2_1 start-tb-node.sh Up 0.0.0.0:49892->7070/tcp,:::49892->7070/tcp, 0.0.0.0:49891->8080/tcp,:::49891->8080/tcp
docker_tb-http-transport1_1 start-tb-http-transport.sh Up 0.0.0.0:49893->8081/tcp,:::49893->8081/tcp
docker_tb-http-transport2_1 start-tb-http-transport.sh Up 0.0.0.0:49894->8081/tcp,:::49894->8081/tcp
docker_tb-js-executor_1 docker-entrypoint.sh start ... Up
docker_tb-js-executor_2 docker-entrypoint.sh start ... Up
docker_tb-js-executor_3 docker-entrypoint.sh start ... Up
docker_tb-js-executor_4 docker-entrypoint.sh start ... Up
docker_tb-js-executor_5 docker-entrypoint.sh start ... Up
docker_tb-lwm2m-transport_1 start-tb-lwm2m-transport.sh Up 0.0.0.0:5685->5685/udp,:::5685->5685/udp
docker_tb-mqtt-transport1_1 start-tb-mqtt-transport.sh Up 0.0.0.0:49896->1883/tcp,:::49896->1883/tcp
docker_tb-mqtt-transport2_1 start-tb-mqtt-transport.sh Up 0.0.0.0:49895->1883/tcp,:::49895->1883/tcp
docker_tb-rule-engine1_1 start-tb-node.sh Up 0.0.0.0:49888->8080/tcp,:::49888->8080/tcp
docker_tb-rule-engine2_1 start-tb-node.sh Up 0.0.0.0:49887->8080/tcp,:::49887->8080/tcp
docker_tb-snmp-transport_1 start-tb-snmp-transport.sh Up
docker_tb-web-ui1_1 docker-entrypoint.sh start ... Up 0.0.0.0:49878->8080/tcp,:::49878->8080/tcp
docker_tb-web-ui2_1 docker-entrypoint.sh start ... Up 0.0.0.0:49881->8080/tcp,:::49881->8080/tcp
docker_zookeeper_1 /docker-entrypoint.sh zkSe ... Up 0.0.0.0:49880->2181/tcp,:::49880->2181/tcp, 2888/tcp, 3888/tcp, 8080/tcp
haproxy-certbot /start.sh Up 0.0.0.0:1883->1883/tcp,:::1883->1883/tcp, 0.0.0.0:443->443/tcp,:::443->443/tcp,
0.0.0.0:7070->7070/tcp,:::7070->7070/tcp, 0.0.0.0:80->80/tcp,:::80->80/tcp,
0.0.0.0:9999->9999/tcp,:::9999->9999/tcp
Below the details of my VM:
cat /etc/os-release
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
Stepping: 7
CPU MHz: 2294.608
BogoMIPS: 4589.21
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 22528K
NUMA node0 CPU(s): 0
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constan
t_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetc
h invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd av
x512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512_vnni md_clear flush_l1d arch_capabilities
free -m
total used free shared buff/cache available
Mem: 7606 7272 117 5 217 94
Swap: 6143 6143 0
Socket(s): 1
Note that you allocated only 1 core and 1 thread for the VM and microservice deployment with 21 services on-board. I think you need at least a minimum of 2 cores and 4 threads for this.
Best regards,
Denys
Socket(s): 1
Note that you allocated only 1 core and 1 thread for the VM and microservice deployment with 21 services on-board. I think you need at least a minimum of 2 cores and 4 threads for this.
Best regards, Denys
Thank you @den1jkee ! Yeah I know that the VM my infrastructure team has provided me with is lacking of HW resources and that's the main reason why it's so slow and takes so long to deploy :( . I'm going to request (hope your comment will help!) if they can increase the resources and check whether performance improves ;)
Apart from this, I would like to take the opportunity to ask you the following question:
We have done tests to deploy Custom Nodes for the Rules Engine (as we have already developed some of them). I have been able to get them to show up already in the WebUI interface, but I have needed to restart several services (as there is no reference to this option in the documentation as it only mention the Monolithic deployment not Microservices as is the case.
Could you tell me which services from below list would be necessary/mandatory to restart for the changes of Custom Nodes to be applied?
Many thanks in advance!
Hi @alexcorvis84,
In your case, restarting tb_core1-2 and tb-rule-engine1-2 will be enough if you mount an external volume for these nodes and extensions. In another case probably you will need to restart all containers with ./docker-stop-services.sh and up ./docker-start-services.sh
I can suggest you use a monolithic type of instance for testing and developing where any actions for developing or testing you can make easier or use docker microservices in basic mode, as an example, you can use this https://github.com/thingsboard/thingsboard-pe-docker-compose
Best regards,
Denys
Hi @den1jkee ,
Thank you so much for your quick support and suggestions.
I'm only pending now just one last question, for which I submitted an issue in the right place. Hope someone could answer :)
Regards.
Update:
We've checked that after waiting ~20' since executing the command ./docker-start-services.sh we are finally able to Login into the platform and the message 503 Service Unavailable is no longer displayed.
VM resourcers (CPU/RAM) stucks at 100% for a while since services start, but afterwards resources are freed up.
So, @star7128, my advise is to wait for a while and check if it works for you as well.
Please, keep me updated ;-)
We can run tb msa via docker-compose with kafka based on 2 cores, 16G memory.
