Wednesday, July 24, 2013

How to grep a log file in realtime / continuously

When hunting down errors and debugging problems that plague a system. Your log file is your best friend. It lets you know what's going on in your application if you've really put the effort into logging all the idiosyncrasies  in your code.

One of my friends once told me: "You have to love your logger" and there's really on true-er statement than that.

Now it's easy to grep a log and look around for what you want. But continuously monitoring a log often poses a problem to people new to the debugging scene. They try out this first:

tail -f logfile | grep "pattern" 

And then they wait 4 days for something to show up. Why? Because of line buffering and the output of tail being buffered before being piped to grep. So how do you get around that? 

I've seen solutions like this:

tail -f logfile | grep --line-buffered "pattern"

but sometimes, depending on the system, they buffer as well and you have to wait.

To get continuous. real time logging you can employ the following shell script fu:

tail -f logfile | while read line ; do echo "$line"| grep "pattern" ; done

No comments:

Post a Comment