At $dayjob I’ve got several front end web servers in load balancing, and I needed a good solution to consolidate all of those log files. Traditionally this is done by moving the files periodically to a central host and then crunching them down. This did not seem ideal to me because if you are generating gigabytes of log a day, it takes a while to move them around. Instead, I wanted a way to consolidate the logs in realtime. Taking some hints from Theo Schlossnagle, I have a solution working that consists of:
- Spread toolkit – handles talking to the network
- mod_log_spread – let’s Apache hosts log to the Spread ring
- spreadlogd - reads from the Spread ring and writes to a consolidated log file
So far it’s working well. There is a slight but acceptable CPU hit from spread. Getting everything installed was a pain in the ass–there was quite a bit of Googling involved to get all the pieces working, so it’s not a task for the faint of heart.
What are some other solutions?
4 responses so far ↓
Michael // January 10, 2008 at 7:33 pm |
I have been attempting to play around with spread over the past day. “quite a bit of Googling” was pretty accurate. Did you get mod_log_spread to compile and install for Apache 2? I seem to run into a problem when compiling both the original mod_log_spread and the alpha version that was ported for Apache 2. Any recommendations or references? Thanks!
John Herren // January 10, 2008 at 8:01 pm |
@Michael: It’s been so long ago that I don’t remember what I had to tweak. I’m thinking I got the beta Apache2 version working, but I had to tweak the make file. I also remember modifying spreadlogd’s source file to support log files over 2GB. George and Theo at Omni TI are nice guys, so you can probably track them down or find the appropriate mailing list if you’re still struggling. I’ve built plenty of software in my day, but I haven’t done a lot of C development; these packages were particularly tough for me to get up and running.
John // February 22, 2008 at 3:31 am |
I have found that mod_log_spread2 does not function ootb with apache 2.2.4 ubuntu. After building and installing, I was getting parse errors on the log format strings. I solved the problem by changing the CustomLog config keyword to SpreadCustomLog. I am getting correctly formatted log data on the logging system.
Rob // May 13, 2009 at 1:44 pm |
@John: Could you post a link to your working config? I’m trying to get this working, too.