Monitor your systems with logstash and Graylog2

I begin a series on configuring a secure local CentOS 6 installation of logstash and Graylog2 which are open source alternatives to commercial packages such as Splunk or Logscape. Using these tools you can see detailed and historical machine data such as WWW and FTP logs, as well as aggregate system logs.

We’ll also make use of a number of other open source tools in our installation, MongoDB, Java Service Wrapper, elasticsearch, and Ruby on Rails running on Apache via Passenger.

Our initial configuration will be a stand alone system, but in future posts we’ll show you how to expand our monitoring of remote systems.

While these tools are not fully mature, and you might not want to use them in an enterprise environment, they are almost certainly an improvement over the method you currently use (I suspect you are not fully reviewing your log files daily).

Our configuration goals in building this test system shall be:

  • Consistent Installation and configuration
  • Running the processes as an unprivileged user
  • Ability to scale to securely monitor other remote systems

So let’s get started!