英文:
How to get Log Aggregation in code by yarnClient
问题
我正在遵循这个关于使用Java API创建YarnApp的示例。
https://github.com/hortonworks/simple-yarn-app
工作得很好，但日志仅在执行时存在，执行后日志消失。
我怎样才能通过代码捕获这个日志？或者也许启用一个选项？
英文:
I'm following this example about create YarnApp by java API.
https://github.com/hortonworks/simple-yarn-app
Works fine, but, the log exists only execution, after it the log gone.
How I can caught this by code ? or maybe enable one option?
答案1
得分: 0
使用LogCliHelpers可以在应用程序完成后通过应用程序ID查找日志：
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.hadoop.yarn.api.records.ApplicationId;
import org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext;
import org.apache.hadoop.yarn.client.api.YarnClientApplication;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.exceptions.YarnException;
import org.apache.hadoop.yarn.logaggregation.LogCLIHelpers;
import java.io.IOException;
import java.io.PrintStream;
public static void getLogs(YarnConfiguration conf, YarnClientApplication app) throws IOException, YarnException {
ApplicationSubmissionContext appContext =
app.getApplicationSubmissionContext();
ApplicationId appId = appContext.getApplicationId();
LogCLIHelpers logCLIHelpers = new LogCLIHelpers();
logCLIHelpers.setConf(conf);
FileSystem fs = FileSystem.get(conf);
Path logFile = new Path("/path/to/log/file.log");
fs.create(logFile, false);
try (PrintStream printStream = new PrintStream(logFile.toString())) {
logCLIHelpers.dumpAllContainersLogs(appId, UserGroupInformation.getCurrentUser().getShortUserName(), printStream);
}
}
英文:
You can find logs using LogCliHelpers by application id after application had finished:
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.hadoop.yarn.api.records.ApplicationId;
import org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext;
import org.apache.hadoop.yarn.client.api.YarnClientApplication;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.exceptions.YarnException;
import org.apache.hadoop.yarn.logaggregation.LogCLIHelpers;
import java.io.IOException;
import java.io.PrintStream;
public static void getLogs(YarnConfiguration conf, YarnClientApplication app) throws IOException, YarnException {
ApplicationSubmissionContext appContext =
app.getApplicationSubmissionContext();
ApplicationId appId = appContext.getApplicationId();
LogCLIHelpers logCLIHelpers = new LogCLIHelpers();
logCLIHelpers.setConf(conf);
FileSystem fs = FileSystem.get(conf);
Path logFile = new Path("/path/to/log/file.log");
fs.create(logFile, false);
try (PrintStream printStream = new PrintStream(logFile.toString())) {
logCLIHelpers.dumpAllContainersLogs(appId, UserGroupInformation.getCurrentUser().getShortUserName(), printStream);
}
}
专注分享java语言的经验与见解，让所有开发者获益！
评论