Class.forName(DRIVER);
Connection = DriverManager.getConnection(CONNECTION_URL, USERNAME, PASSWORD);
Response = Connection.createStatement();
ResultSet = Response.executeQuery(query);
I can see the application details in yarn ui. But now i want to get the application id for this job through java code, Is it possible to do so? If yes, then how?
AFAIK the short answer is: not in older versions of Hive; possibly with recent versions, which let you retrieve some logs, which may contain the YARN ID.
Starting with Hive 0.14 you can set up HiveServer2 to publish the execution logs for the current Statement; and in your client code you can use a Hive-specific API to fetch these logs (asynchronously just like Beeline client does, or just once when execution is over).
Quoting Hive documentation
Starting with Hive 0.14.0, HiveServer2 operation logs are available
for Beeline clients. These parameters configure logging:
hive.server2.logging.operation.enabled
hive.server2.logging.operation.log.location
hive.server2.logging.operation.verbose (Hive 0.14 to 1.1)
hive.server2.logging.operation.level (Hive 1.2 onward)
Hive 2.0 adds the support of logging queryId and sessionId to HiveServer2 log file (...)
The source code for HiveStatement shows several non-JDBC methods such as getQueryLog and hasMoreLogs -- also getYarnATSGuid for Hive 2+ and other stuff for Hive 3+.
Here is the link to the "master" branch on GitHub, switch to whichever version you are using (possibly an old 1.2 for compatibility with Spark).
For a dummy demo about how to tap the "Logs" methods, have a look at that SO post with a snippet.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
