Java实时日志大文件分析

日志分析时经常会遇到这个问题：一个正在增长中的大文件，要分析新增的每一行的固定内容进行统计并展示啥的。用java的RandomAccessFile盯住文件的末尾，是一个不错的选择，下面的实现，是一个高效的java tail工具，实现类似linux的tail工具的功能。用来做大日志文件的实时分析，是一个不错的选择。

public class LogFileTailer extends Thread {
private Logger logger = LoggerFactory.getLogger(LogFileTailer.class);

/**
* How frequently to check for file changes; defaults to 5 seconds
*/
private long sampleInterval = 5000;

/**
* The log file to tail
*/
private File logfile;

/**
* Defines whether the log file tailer should include the entire contents of
* the exising log file or tail from the end of the file when the tailer
* starts
*/
private boolean startAtBeginning = false;

/**
* Is the tailer currently tailing?
*/
private boolean tailing = false;

/**
* Set of listeners
*/
private Set listeners = new HashSet();

/**
* Creates a new log file tailer that tails an existing file and checks the
* file for updates every 5000ms
*/
public LogFileTailer(File file) {
this.logfile = file;
}

/**
* Creates a new log file tailer
*
* @param file The file to tail
* @param sampleInterval How often to check for updates to the log file
* (default = 5000ms)
* @param startAtBeginning Should the tailer simply tail or should it
* process the entire file and continue tailing (true) or simply
* start tailing from the end of the file
*/
public LogFileTailer(File file, long sampleInterval, boolean startAtBeginning) {
this.logfile = file;
this.sampleInterval = sampleInterval;
}

public void addLogFileTailerListener(LogFileTailerListener l) {
this.listeners.add(l);
}

public void removeLogFileTailerListener(LogFileTailerListener l) {
this.listeners.remove(l);
}

protected void fireNewLogFileLine(String line) {
for (Iterator i = this.listeners.iterator(); i.hasNext();) {
LogFileTailerListener l = (LogFileTailerListener) i.next();
l.newLogFileLine(line);
}
}

public void stopTailing() {
this.tailing = false;
}

public void run() {
long filePointer = 0;

if (this.startAtBeginning) {
filePointer = 0;
} else {
filePointer = this.logfile.length();
}

try {
this.tailing = true;
RandomAccessFile file = new RandomAccessFile(logfile, “r”);
while (this.tailing) {
long fileLength = this.logfile.length();
if (fileLength < filePointer) {
file = new RandomAccessFile(logfile, “r”);
filePointer = 0;
}
if (fileLength > filePointer) {
file.seek(filePointer);
String line = file.readLine();
while (line != null) {
this.fireNewLogFileLine(line);
line = file.readLine();
}
filePointer = file.getFilePointer();
}
sleep(this.sampleInterval);
}
file.close();
} catch (IOException e) {
logger.error(“IO ERROR:”, e);
} catch (InterruptedException e) {
logger.error(“InterruptedException:”, e);
}

}
}

使用起来很简单，只需要实现一个LogFileTailerListener后加入到tailer，这个tailer会去自动去执行。

类似：

public class Tail implements LogFileTailerListener {
/**
* The log file tailer
*/
private LogFileTailer tailer;

/**
* Creates a new Tail instance to follow the specified file
*/
public Tail(String filename) {
tailer = new LogFileTailer(new File(filename), 1000, false);
tailer.addLogFileTailerListener(this);
tailer.start();
}

/**
* A new line has been added to the tailed log file
*
* @param line The new line that has been added to the tailed log file
*/
public void newLogFileLine(String line) {
System.out.println(line);
}

/**
* Command-line launcher
*/
public static void main(String[] args) {
if (args.length < 1) {
System.out.println(“Usage: Tail “);
System.exit(0);
}
Tail tail = new Tail(args[0]);
}
}

上面的代码出处：http://www.informit.com/guides/content.aspx?g=java&seqNum=226，我(54chen)对其进行了一些修改，保障出现异常时不会出现死循环。

下载修改后的可用的代码： http://ishare.iask.sina.com.cn/f/15999043.html

原创文章如转载，请注明：转载自五四陈科学院[http://www.54chen.com]

Posted by 54chen java