How to Install a Production Ready Naemon Monitoring Server

Naemon is the new monitoring suite that aims to be fast, stable and innovative while giving you a clear view of the state of your network and applications.This is a replacement for nagios server and very stable and uses an amazing dashboard called thruk. Setting up naemon is very easy and has live status inbuild .In this tutorial, I will show you how to install and configure naemon in a production environment along with pnp4nagios for graphs. Naemon is very stable, I will show you how to maintain the configs in a large scale environment.

In this tutorial, I will be using four Servers

  • 192.168.1.170 - Naemon Server - centos 7
  • 192.168.1.21 - nrpe client - centos 6
  • 192.168.1.182 - nrpe client - ubuntu
  • 192.168.1.75 - nscp client - windows server 2012R2

Installation of Naemon Server

I am using a CentOS 7 host with a minimal installation, SELinux disabled and firewalld enabled.

[root@naemon ~]# sestatus
SELinux status:                 disabled
[root@naemon ~]#

Installing the repo:

[root@naemon ~]# yum -y install https://labs.consol.de/repo/stable/rhel7/x86_64/labs-consol-stable.rhel7.noarch.rpm

enabling the epel repo:

yum -y install epel-release

Installing the Naemon package:

[root@naemon ~]# yum install naemon*

Naemon uses the same Nagios plugin and nrpe to monitor hosts. We will now install them:

[root@naemon ~]# yum install nagios-plugins nagios-plugins-all nagios-plugins-nrpe nrpe -y

We will add the firewall rules:

   
[root@naemon ~]# firewall-cmd --add-service=http --permanent
success
[root@naemon ~]# firewall-cmd --add-service=https --permanent
success
[root@naemon ~]# firewall-cmd --reload
success
[root@naemon ~]# firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: eno16777736
  sources:
  services: dhcpv6-client http https ssh
  ports:
  protocols:
  masquerade: no
  forward-ports:
  sourceports:
  icmp-blocks:
  rich rules:

[root@naemon ~]#

    

Installation of pnp4nagios.

yum install gcc-c++ rrdtool perl-Time-HiRes perl-rrdtool php-gd php php-cli wget

Downloading pnp4nagios.

[root@naemon ~]# wget https://svwh.dl.sourceforge.net/project/pnp4nagios/PNP-0.6/pnp4nagios-0.6.25.tar.gz
    [root@naemon ~]# tar -zxvf pnp4nagios-0.6.25.tar.gz
    [root@naemon ~]# cd pnp4nagios-0.6.25/
[root@naemon pnp4nagios-0.6.25]# ./configure --with-nagios-user=naemon --with-nagios-group=naemon
[root@naemon pnp4nagios-0.6.25]# make all
[root@naemon pnp4nagios-0.6.25]# make fullinstall
[root@naemon pnp4nagios-0.6.25]# make install-webconf
[root@naemon pnp4nagios-0.6.25]# make install-config
[root@naemon pnp4nagios-0.6.25]# make install-init
    

Restarting and enabling the service.

    [root@naemon ~]# systemctl enable httpd && systemctl restart httpd
[root@naemon ~]# systemctl enable naemon && systemctl restart naemon
[root@naemon ~]# systemctl enable thruk && systemctl restart thruk
[root@naemon ~]# systemctl enable npcd && systemctl restart npcd
    

you should be able to access the URL from http://192.168.1.170/thruk

Thruk interface

The default username is thrukadmin, password is thrukadmin.

In case you want to change the thrukadmin password, run:

[root@naemon ~]# htpasswd /etc/thruk/htpasswd thrukadmin

Installation of Monitoring Client to monitor the nodes

Installation of nrpe clients for monitoring Linux hosts

192.168.1.21 - nrpe client - centos 6

Installation of the Nagios plugin and nrpe client:

[root@centos6 ~]# yum install nagios-plugins nagios-plugins-all nagios-plugins-nrpe nrpe -y

Take a backup of the nrpe.cfg:

[root@centos6 ~]# cp /etc/nagios/nrpe.cfg /etc/nagios/nrpe.cfg.bkp

Replace the content of the nrpe.conf file with the below content:

[root@centos6 ~]# cat /etc/nagios/nrpe.cfg
log_facility=daemon
pid_file=/var/run/nrpe/nrpe.pid
nrpe_user=nrpe
nrpe_group=nrpe
allowed_hosts=127.0.0.1,192.168.1.170
dont_blame_nrpe=1
debug=0
command_timeout=60
connection_timeout=300
include_dir=/etc/nrpe.d/

[root@centos6 ~]#
    

Under allowed_host add the naemon server ipaddress here it is 192.168.1.170.

Now we need to restart the nrpe service:

	[root@centos6 ~]# service nrpe restart
	[root@centos6 ~]# chkconfig nrpe on
	

192.168.1.182 - nrpe client - ubuntu

root@ubuntu:~# apt-get install nagios-nrpe-server nagios-plugins -y

Take a backup of the nrpe.cfg:

[root@ubuntu ~]# cp /etc/nagios/nrpe.cfg /etc/nagios/nrpe.cfg.bkp

Replace the content of the nrpe.conf file with the below content:

root@ubuntu:~# cat /etc/nagios/nrpe.cfg
log_facility=daemon
pid_file=/var/run/nagios/nrpe.pid
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127.0.0.1,192.168.1.170
dont_blame_nrpe=1
debug=0
command_timeout=60
connection_timeout=300
include_dir=/etc/nagios/nrpe.d/
root@ubuntu:~#
    

Now we need to restart the nrpe service:

	root@ubuntu:~# systemctl restart nagios-nrpe-server.service
	root@ubuntu:~# systemctl enable nagios-nrpe-server.service
	

Installation of Nscp on Windows

192.168.1.75 - nscp client - windows server 2012R2

Download nscp from this website:

https://www.nsclient.org/download/

Once downloaded, please follow the installation screenshots

Choose generic.

Choose complete:

Choose the default path:

Now, under allowed hosts, please add the naemon host. Here in our setup it is 192.168.1.170.

Make sure you give a complex password. For this example, I have used the password as "howtoforge". Next check enable nsclient and nrpe.

Once the installation is complete, please make sure the service is up and running.

We will now validate from naemon host, to make sure we are able to reach it.

Validate nrpe and nscp from Naemon Server to check the connectivity

[root@naemon ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.1.182
NRPE v2.15
[root@naemon ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.1.21
NRPE v2.15
[root@naemon ~]#
	
[root@naemon ~]# /usr/lib64/nagios/plugins/check_nt -H 192.168.1.75 -p 12489 -v CLIENTVERSION -s 'howtoforge'
NSClient++ 0.5.0.62 2016-09-14
[root@naemon ~]#

	

Monitoring of Servers

We will monitor the server's performance in this tutorial.

In a production environment, we will need to monitor the CPU , memory and harddisk utilization, along with the host alive check.

Adding the following lines in nrpe conf /etc/nrpe.d/base.cfg

192.168.1.21 - nrpe client - centos 6

  [root@centos6 ~]# cat /etc/nrpe.d/base.cfg
command[check_disks]=/usr/lib64/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
command[check_disk]=/usr/lib64/nagios/plugins/check_disk -X nfs -X cifs -X none -X tmpfs -w $ARG1$ -c $ARG2$
command[memory]=/usr/lib64/nagios/plugins/check_memory -w $ARG1$ -c $ARG2$
command[load]=/usr/lib64/nagios/plugins/check_load -w $ARG1$ -c $ARG2$
command[check_total_procs]=/usr/lib64/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$
command[check_procs_c]=/usr/lib64/nagios/plugins/check_procs -c $ARG1$ -C $ARG2$
command[check_procs_a]=/usr/lib64/nagios/plugins/check_procs -c $ARG1$ -a $ARG2$
  

Save and restart the file, then restart nrpe.

[root@centos6 ~]# /etc/init.d/nrpe restart
Shutting down nrpe:                                        [FAILED]
Starting nrpe:                                             [  OK  ]
[root@centos6 ~]#
 

Memory plugin will not be available as default plugin. We will use the below plugin.

 [root@centos6 ~]# cd  /usr/lib64/nagios/plugins/
 [root@centos6 plugins]# vim check_memory

Add the below content.

#! /usr/bin/perl -w
##Included Perf data
#The nagios plugins come with ABSOLUTELY NO WARRANTY. You may redistribute
#copies of the plugins under the terms of the GNU General Public License.
#
use strict;
use Getopt::Long;

my ($opt_f, $opt_h, $opt_c, $opt_w);
my ($result, $message, $ram_percent);

my $PROGNAME="check_memory";

$opt_f = "";

Getopt::Long::Configure('bundling');
GetOptions(
        "h"   => \$opt_h, "help"        => \$opt_h,
        "c=s" => \$opt_c, "Warning_Percentage"  => \$opt_c,
        "w=s" => \$opt_w, "Critical_Percentage" => \$opt_w,
);

if ($opt_h) {
        &print_help();
        exit 0;
}

$opt_w = shift unless ($opt_w);
$opt_c = shift unless ($opt_c);

if (! $opt_c && ! $opt_w) {
        print "Warning and Critical must be specified\n";
        exit 3;
}
if ($opt_c <= $opt_w){
        print "Critical must be greater than Warning CRITICAL: $opt_c WARNING: $opt_w\n";
        exit 3;
}



$result = 'OK';
open(FREE_M, "free -m|");
while (my $line = ){
        if ($line =~ /^Mem/){
                my ($junk,$total_mem,$used_mem,$free_mem,$shared_mem,$buffered_mem,$cached_mem)=split(/\s+/,$line);
                $ram_percent=int(($total_mem-($free_mem+$cached_mem+$buffered_mem))/$total_mem*100*100)/100;

        }
}
close(FREE_M);

if ($ram_percent > $opt_c){
print "Memory usage is: CRITICAL: ${ram_percent}% used | used=$ram_percent;$opt_w;$opt_c\n";
exit 2;
}elsif ($ram_percent > $opt_w) {
print "Memory usage is: WARNING: ${ram_percent}% used | used=$ram_percent;$opt_w;$opt_c\n";
exit 1;
}
else{
print "Memory usage is: $result: ${ram_percent}% used | used=$ram_percent;$opt_w;$opt_c\n";
exit 0;
}

sub print_usage () {
        print "Usage:\n";
        print "  $PROGNAME -f \n";
        print "  $PROGNAME -c \n";
        print "  $PROGNAME -w \n";
        print "  $PROGNAME [-h | --help]\n";
        print "  $PROGNAME [-V | --version]\n";
}

sub print_help () {
        print << EOF; 
$PROGNAME v1.0.01
The nagios plugins come with ABSOLUTELY NO WARRANTY. You may redistribute
copies of the plugins under the terms of the GNU General Public License.
EOF
        &print_usage();
        print "\n";
        exit 3;
}


Download link check_memory plugin

You have to give execute permissions to the file:

  [root@centos6 plugins]# chmod 755 check_memory

Adding the following lines in nrpe conf /etc/nagios/nrpe.d/base.cfg

192.168.1.182 - nrpe client - ubuntu

Here the file path is different:

The plugin folder is lib and not lib64.

  root@ubuntu:~# cat /etc/nagios/nrpe.d/base.cfg
command[check_disks]=/usr/lib/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
command[check_disk]=/usr/lib/nagios/plugins/check_disk -X nfs -X cifs -X none -X tmpfs -w $ARG1$ -c $ARG2$
command[memory]=/usr/lib/nagios/plugins/check_memory -w $ARG1$ -c $ARG2$
command[load]=/usr/lib/nagios/plugins/check_load -w $ARG1$ -c $ARG2$
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$
command[check_procs_c]=/usr/lib/nagios/plugins/check_procs -c $ARG1$ -C $ARG2$
command[check_procs_a]=/usr/lib/nagios/plugins/check_procs -c $ARG1$ -a $ARG2$
  

Add the same check_memory script in /usr/lib/nagios/plugins.

  root@ubuntu:~# vim /usr/lib/nagios/plugins/check_memory
root@ubuntu:~# chmod 755 /usr/lib/nagios/plugins/check_memory
root@ubuntu:~# systemctl restart nagios-nrpe-server.service
root@ubuntu:~#

Configuration on Naemon Server

Login as naemon user:

[root@naemon ~]# su - naemon
Last login: Thu Mar  9 17:11:39 CET 2017 on pts/0
-sh-4.2$

Change the admin mail in /etc/naemon/naemon.cfg

........
........
[email protected]
[email protected]
........
........

Now we will configure the custom commands to do the check in /etc/naemon/conf.d

Create a custom commands called custom_commands.cfg with below content.

$ vim /etc/naemon/conf.d/custom_commands.cfg
## Host Alive check ##
# command 'check_icmp Host Alive'
define command{
    command_name                   host_alive
    command_line                   $USER1$/check_icmp -H $HOSTADDRESS$ -n 1 -w 3000.00,80% -c 5000.00,100%
    }
### Service Command ##
# command 'check_nrpe_disks'
define command{
    command_name                   check_nrpe_disks
    command_line                   $USER1$/check_nrpe -t 55 -u -H $HOSTADDRESS$ -c check_disks -a $ARG1$ $ARG2$ $ARG3$
    }

# command 'nrpe'
define command{
    command_name                   nrpe
    command_line                   $USER1$/check_nrpe -t 55 -u -H $HOSTADDRESS$ -c $ARG1$
    }

# NRPE With Arguements
define command{
    command_name                   check_nrpe_a
    command_line                   $USER1$/check_nrpe -t 55 -u -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$
    }
#####################################
# Windows NSClient++ Checks
# command 'check_nt'
define command{
    command_name                   nt
    command_line                   $USER1$/check_nt -H $HOSTADDRESS$ -p 12489 -v $ARG1$ $ARG2$ -s $USER4$
    }

For safety purpose save the nscp clients password in /etc/naemon/resource.cfg

...........
...........
$USER4$=howtoforge
...........
............

Now we need to define the host config.

We create a file called linux.cfg.

$ vim /etc/naemon/conf.d/linux.cfg
define host {
  host_name                      centos6.sunil.cc
  alias                          Centos
  address                        192.168.1.21
  use                            linux-server
  hostgroups                     linux-servers
}
define host {
  host_name                      ubuntu.sunil.cc
  alias                          Ubuntu
  address                        192.168.1.182
  use                            linux-server
  hostgroups                     linux-servers
}

We do the same for the windows host.

$ vim /etc/naemon/conf.d/windows.cfg
define host {
  host_name                      ad.sunil.cc
  alias                          My Windows Server
  address                        192.168.1.75
  use                            windows-server
}

define hostgroup {
  hostgroup_name                 windows-servers
  alias                          Windows Servers
}

Now we need to define the service config.

$ vim linux_services.cfg
define service {
  service_description            Disks Status
  hostgroup_name                  linux-servers
  use                            generic-service                       
  check_command                  check_nrpe_a!check_disk!15%!5%
}
define service{
    use                            generic-service
    hostgroup_name                 linux-servers
    service_description            Load Average
    check_command                  check_nrpe_a!load!30,25,20!45,40,35
    }
define service{
    use                            generic-service
    hostgroup_name                 linux-servers
    service_description            Memory Usage
    check_command                  check_nrpe_a!memory!80!90
    }

For Windows, we create a similar config.

$ vim window_services.cfg
define service{
    use                            generic-service
    hostgroup_name                 windows-servers
    service_description            Disk - C:
    check_command                  nt!USEDDISKSPACE!-l c -w 80 -c 90
     }
define service{
    use                            generic-service
    hostgroup_name                 windows-servers
    service_description            Load Average
    check_command                  nt!CPULOAD!-l 60,90,95
    }
define service{
    use                            generic-service
    hostgroup_name                 windows-servers
    service_description            Memory Usage
    check_command                  nt!MEMUSE!-w 90 -c 95
    }

Now we will change the contact email address.

$ vim contacts.cfg
define contact {
  contact_name                   naemonadmin                         ; Short name of user
  alias                          Naemon Admin                        ; Full name of user
  use                            generic-contact                     ; Inherit default values from generic-contact template (defined above)
  email                          [email protected]             ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
}

We now remove the unnecessary configs which are not required.

$ mv switch.cfg switch.cfg.bkp
$ mv printer.cfg printer.cfg.bkp

To validate the configs.

$ naemon -v /etc/naemon/naemon.cfg

Naemon Core 1.0.6-pkg
Copyright (c) 2013-present Naemon Core Development Team and Community Contributors
Copyright (c) 2009-2013 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
License: GPL

Website: http://www.naemon.org
Reading configuration data...
   Read main config file okay...
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
        Checked 20 services.
        Checked 4 hosts.
        Checked 1 contacts.
        Checked 2 host groups.
        Checked 0 service groups.
        Checked 1 contact groups.
        Checked 29 commands.
        Checked 5 time periods.
        Checked 0 host escalations.
        Checked 0