[Bug]amazoncorretto jdk21 cannot obtain data · apache/skywalking · Discussion #12260

link管理
链接快照平台
输入网页链接，自动生成快照
标签化管理网页链接
What happened

I deployed skywalking OAP Server version 10.0.0 , agent is version 9.2 , and Java runtime is amazoncorretto jdk21 version. Now that the service has been successfully registered to skywalking, some data can be seen through the UI, but most of it is missing. I didn’t see any error logs. I would like to ask what is the reason for the lack of data. I also want to confirm whether it is compatible and supports amazoncorretto jdk21.
What you expected to happen

I hope to be able to collect jdk21 data normally
How to reproduce

The problem can be reproduced by running a java program through the amazoncorretto:21.0.1-al2023-headless image, and adding the skywalking agent, version 9.2, and the OAP Server is 10.0.0
Anything else

Because I can't see the error log, I can't locate the specific problem. I also hope to give some suggestions to solve this problem.
Screenshots of relevant indicators are as follows:
At the same time, I also found that the indexes in ES are much less than before.
Are you willing to submit a pull request to fix on your own?

Yes I am willing to submit a pull request on my own!
Code of Conduct

I agree to follow this project's Code of Conduct
I downloaded apache-skywalking-java-agent-9.2.0.tgz directly. After decompression, I specified -javaagent:/tmp/skywalking-agent/skywalking-agent.jar through JAVA_TOOL_OPTIONS to start it. No plug-ins were added or deleted, and then I found the following error in the logs directory。
Apart from the error below, I have not observed any other problems. Do you have any other troubleshooting ideas?
ERROR 2024-05-28 16:59:07.554 http-nio-8080-exec-9 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 16:59:07.554 http-nio-8080-exec-1 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 17:00:07.556 http-nio-8080-exec-10 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 17:00:07.556 http-nio-8080-exec-2 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 17:01:07.556 http-nio-8080-exec-4 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 17:01:07.559 http-nio-8080-exec-3 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 17:02:07.556 http-nio-8080-exec-5 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
ERROR 2024-05-28 17:02:07.558 http-nio-8080-exec-6 InstMethodsInter : class[class org.apache.catalina.core.StandardHostValve] after method[invoke] intercept failure 
java.lang.ClassCastException: class org.apache.catalina.connector.Response cannot be cast to class javax.servlet.http.HttpServletResponse (org.apache.catalina.connector.Response and javax.servlet.http.HttpServletResponse are in unnamed module of loader org.springframework.boot.loader.launch.LaunchedClassLoader @52b6319f)
        at org.apache.skywalking.apm.plugin.tomcat78x.TomcatInvokeInterceptor.afterMethod(TomcatInvokeInterceptor.java:93)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:97)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        Maybe you should search the issue or use ASK AI at skywalking.apache.org
How to resolve unnamed module issue?
To resolve the unnamed module issue in SkyWalking, you can follow these steps:
Add JVM Options:

You can add JVM options to export the necessary modules. For example, if you encounter an issue where the java.base module does not export jdk.internal.misc, you can add the following JVM option:
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED




    

Avoid Using Internal APIs:

Try to avoid using internal APIs such as sun.* or jdk.internal.*. Look for alternative ways to achieve the same functionality using public APIs.
Update Documentation:

If you are a developer or contributor to SkyWalking, consider updating the relevant documentation to provide solutions and example code. You can refer to the SkyWalking documentation for more details:
SkyWalking Java Agent Bootstrap Plugins
Use --illegal-access Option:

In Java 9-15, the default --illegal-access mode is permit, which allows illegal access to internal APIs. However, in Java 16 and later, this mode is set to deny. You can temporarily resolve the issue by adding the following JVM option:
--illegal-access=permit
Here are some relevant discussions and solutions from the SkyWalking GitHub issues:
Issue 8133 mentions adding the JVM option:
User can add jvm option: `--add-exports java.base/sun.net.www=ALL-UNNAMED`
Issue 7419 includes error logs and potential solutions:
java.lang.IllegalAccessException: class org.apache.skywalking.apm.dependencies.io.netty.util.internal.PlatformDependent0$6 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @2dc54ad4
By following these steps, you should be able to resolve the unnamed module issue. If the problem persists, you can refer to the SkyWalking official documentation or seek further assistance from the community.