
The NetMRG Reference

Brady Alleman

Douglas E. Warner

   Copyright © 2003-2004 Brady Alleman and Douglas E. Warner

   This project is licensed under the terms of the MIT License.

   Permission  is  hereby  granted, free of charge, to any person
   obtaining a copy of this software and associated documentation files
   (the "Software"), to deal in the Software without restriction,
   including without limitation the rights to use, copy, modify, merge,
   publish, distribute, sublicense, and/or sell copies of the Software,
   and to permit persons to whom the Software is furnished to do so,
   subject to the following conditions:

   The above copyright notice and this permission notice shall be
   included in all copies or substantial portions of the Software.

   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
   MERCHANTABILITY,   FITNESS   FOR   A  PARTICULAR  PURPOSE  AND
   NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
   BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
   ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
   CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
   SOFTWARE.

   All product names are property of their respective owners. Such
   names  are  used  for identification purposes only and are not
   indicative  of endorsement by or of any company, organization,
   product, or platform.
     _________________________________________________________

   Table of Contents
   I. Overview

        1. Introduction
        2. Revision History

   II. Concepts

        3. System Structure
        4. Monitoring and Reporting Concepts
        5. Graphing Concepts

   III. Installation

        6. Software Requirements
        7. Satisfying Requirements
        8. Installation Procedure
        9. Upgrade Procedure
        10. Basic Tutorial
        11. Graph Tutorial
        12. Interface and Disk Cache Tutorial

   IV. Feature Guide

        13. External Authentication
        14. Workday Highlighting

   V. Reference

        I. Manual Pages

              netmrg-gatherer -- Gathers data from devices.

        II. Web Interface Descriptions

              Monitoring -- Items associated with setting up gathering
                      of information

              Groups -- A container for a collection of devices
              Devices -- A container for a collection of sub-devices
              Sub Devices -- A container for a collection of monitors
              Monitors -- A container for a collection of tracked
                      values

              Events -- A container for a collection of actions to take
                      based on monitor values

              Conditions -- A collection of paramenters to trigger an
                      Event

              Reporting -- Items whose purpose is to display and report
                      information to the user.

              Graphing -- Where graphs (compositions of one or more
                      monitors) are created.

              Tests -- Tests represent actions taken to gather data.
              Notifications -- Configuration for notifying users when
                      events occur.

              Admin -- Administrative functions not related to the
                      reporting or gathering of information.

              Help -- How to get assistance with NetMRG.

   15. Contact Information

I. Overview

   Table of Contents
   1. Introduction
   2. Revision History
     _________________________________________________________

Chapter 1. Introduction

   Welcome to NetMRG! NetMRG is a database-driven network monitoring
   and graphing tool for use on Linux systems. It offers an intuitive
   web-based configuration, customized monitoring capabilities for
   unusual situations, a fast multi-threaded polling application, and
   brilliant graphs rendered by RRDTOOL.

   This document aims to guide you swiftly through the installation and
   configuration of NetMRG, as well as act as a reference for those
   with existing installations.
     _________________________________________________________

Chapter 2. Revision History

     * 0.18.2
       2004.11.09
       Minor Bug Fixes; Fixed some escaping problems on the interface
       cache page, disabled the 'quickprint' Net-SNMP option for better
       compatibility, fixed the snmp-failure test, fixed an SVO-user
       view issue, and added failover for duplicate ifNames when adding
       interfaces.
     * 0.18.1
       2004.11.01
       Minor Bug Fixes; Fixed a small bug where the gatherer would not
       register negative values.
     * 0.18
       2004.10.25
       Major Bug Fixes; Fixed an issue preventing form submissions on
       some  browsers,  fixed PHP string escaping problems, fixed
       inability to create inverted stacked graphs, and fixed parameter
       substituion  and documentation regarding notifications and
       responses. Added better support on FreeBSDs (4.8-5.2) and x86-64
       architectures (Linux and FreeBSD), the ability to multiply
       summed values, MySQL timeouts, simple value parsing (to get the
       numeric content out of quoted numbers, etc), graphic icons for
       most actions, support for a default external auth user (with
       regards to permissions), and cleaned up the gatherer build
       process output.
     * 0.17
       2004.09.13
       Minor Feature Enhancements; Added colorized gatherer output,
       checkbox support on multiple pages, the ability to disable
       users, fixed a potential SQL-injection bug, updated navigation
       on group pages, updated tutorials.
     * 0.16
       2004.06.10
       Minor Bugfixes and Feature Enhancements; fixed a user management
       bug and several bugs related to 'single view only' users not
       being able to see graphs, added the ability to ignore SNMP
       uptime checks for devices (permitting the monitoring of broken
       devices), added the ability to do an SNMP walk of an OID (useful
       for the Windows SNMP agent), enhanced group/device management,
       cleaned up autoconf (now supports FC2), and added experimental
       syslog and daemon support in the gatherer.
     * 0.15
       2004.05.20
       Minor Bugfixes and Feature Enhancements; fixes for various db
       errors (including the infamous one on snmp_cache_view.php),
       better  cross-platform compilation, select all/none/invert
       checkbox support, support for choosing what graph to apply to an
       interface, better support for php < 4.2.0, and some security
       fixes for single-view-only users.
     * 0.14
       2004.03.10
       Minor Bugfixes and Feature Enhancements; escaping issues in
       template dereferencing, broken links on various pages, security
       issues regarding manual SNMP recaching, broken duplication on
       non-cutting-edge MySQL, more parameter substitution in SQL
       tests, better date parsing for advanced graph view, nav-bar and
       group navigation enhancements, advanced view capability for
       monitor-only graphs.
     * 0.13
       2003.12.19
       Minor Bugfixes and Feature Enhancements; slide show ordering,
       previous slide, resume slide show, better formatting of 'sums',
       errors on view editing, fixes to permissions checking, and an
       occassional segfault in the gatherer was fixed.
     * 0.12
       2003.12.15
       Major Feature Enhancents and Bugfixes; added SNMPv2 support,
       fixed a bug that would cause a device to be deleted from all
       groups, fixed session timeout when using external auth, added
       checkbox support to interface/disk cache pages, slide show
       enhancements, fixed permission issue where unauthorized users
       could view/edit test_* pages
     * 0.10
       2003.10.05
       Major bug fix; fixes a db error when trying to view monitors, a
       bug that would always set the width and height to zero when
       creating graphs, and makes conditions with 'rate of change'
       work. Many documentation upgrades.
     * 0.10pre2
       2003.09.15
       This release focuses on adding new features and sqaushing bugs.
       In addition, there is now an 'updater.php' that is required to
       be run between versions. A minor change needs to be made to
       everyone's 'netmrg.xml' to add an 'externalauth' field inside
       the 'website' section.
     * 0.10pre1
       2003.07.28
       Initial publication

II. Concepts

   Table of Contents
   3. System Structure
   4. Monitoring and Reporting Concepts
   5. Graphing Concepts
     _________________________________________________________

Chapter 3. System Structure

   The components of NetMRG.

     * Gatherer
       The Gatherer is a multi-threaded C++ program that interacts with
       the Database and RRDTOOL to gather and store data. The program
       is run by cron, or another scheduling system of your choice, on
       five minute intervals.
     * Web Interface
       The Web Interface consists of several PHP scripts, static HTML
       files and images, as well as CSS for formatting. It interacts
       with the Database for the storage of configuration data and with
       RRDTOOL for the rendering of graphs for presentation.
     * Database
       The Database is a relational database running on MySQL. It is
       utilized by the Web Interface and the Gatherer.
     * RRDTOOL
       RRDTOOL is a graphing system based upon round-robin databases
       (RRDs). It is used for both storing data into the databases and
       creating graphs using the databases.
     * RRDs
       RRDs are the round-robin databases used by RRDTOOL. They are
       individual files that are writable by the Gatherer and readable
       by the Web Interface.
     _________________________________________________________

Chapter 4. Monitoring and Reporting Concepts

     * Groups
       Groups are organizational containers used in NetMRG. They are
       capable  of  containing  Devices, as well as having a View
       associated with them. As they are strictly organizational, the
       Web Interface is the only component to utilize them. They are
       ignored by the Gatherer.
     * Devices
       Devices can be thought of as any physical device on your network
       that will be monitored. They may also be used organizationally
       to  group Monitors that are not associated directly to any
       physcial host. Devices may have, but are not required to have,
       IP  addresses,  SNMP community strings, and so forth. Data
       gathering  for Devices may be explicitly disabled. Devices
       contain Sub-Devices.
     * Sub-Devices
       Sub-Devices are discrete entities on a Device. Examples of these
       are network interfaces, disk drives, and so forth. Sub-Devices
       may  have parameters or variables associated with them for
       identification purposes. For example, a web server has a network
       interface designated "eth0." In NetMRG, the server would have a
       network interface Sub-Device which would have a variable of
       "ifDescr" set to "eth0." Sub-Devices contain Monitors.
     * Monitors
       Monitors represent discrete monitorable quantities. The Gatherer
       performs the Monitor's Test upon the Monitor's Sub-Device. The
       result is then stored in a RRD. Monitors are also associated
       with Graph Items. If NetMRG is being used to track events, a
       Monitor may also contain Events.
     * Tests
       Tests are data gathering operations associated with Monitors.
       There  are four types of tests: Scripts, SNMP Queries, SQL
       Queries, and Internal tests. Script tests call a script and use
       either its return code or its output as data. SNMP Tests perform
       a SNMP GET operation against a device using a supplied OID. SQL
       Tests perform a MySQL query and use a value from a returned row
       as data. Internal tests are frequently used tests built into the
       Gatherer.  They  typically  start  as Script Tests but are
       integrated for the sake of efficiency.
     * Events
       Events contain a group of Conditions and Responses. If the
       Conditions of an Event are met, the appropriate response is
       performed.
     _________________________________________________________

Chapter 5. Graphing Concepts

     * Graphs
       Graphs are the abstraction of a single chart generated by NetMRG
       using RRDTOOL. Graphs have properties such as sizes, a title,
       and vertical labels. They also contain a set of Graph Items
       which represent the separate pieces of data graphed.
          + Custom Graphs
            Custom graphs have "absolute" referencing of data to be
            displayed. They are typically used for graphs that are
            applicable in only one instance, such as a stack graph of
            all Internet links on your network.
          + Templated Graphs
            Template Graphs have "relative" references to data. They
            are used for graphs such as CPU load on a server or traffic
            on  a  network interface. They are applicable in many
            instances. The use of templates allows you to create one
            graph for a set of variables on a Sub-Device and then apply
            that same graph to other Sub-Devices for graphs formatted
            identically. This allows considerable savings of time as
            there is no need to create a Custom Graph for each instance
            of a graph.
     * Graph Items
       Graph Items represent data on a graph. They can represent areas,
       lines, and stacks on Graphs. Each Graph Item is linked to a
       Monitor for dynamic data or given a static value. Options for
       Graph Items include type (such as line, area, stack), color,
       value (either static or dynamic based on a Monitor), and legend
       options.
     * RRDs and Variables
       Graphs  and  variables (subdevice parameters) won't become
       available until after the gatherer runs. The gatherer creates
       RRD  files  that the graphs use to display and inserts the
       variables into the database so they can be used to be displayed
       in the graphs. This can occasionally bite you when you add a new
       parameter and expect it to be displayed immediately in your
       graph.
     * Variables and Parameters
       Subdevice  parameters become variables that can be used in
       various places inside NetMRG. Variables can be used inside
       Tests, Graph Legends and Graph item multipliers.
       Inside  tests,  it allows you to use use the same test for
       multiple  subdevices  by defining a parameter that will be
       substituted in.
       Inside  graph  items, it will allow you to use a graph for
       multiple subdevices.
       Here's a partial list of some variables that will be defined by
       NetMRG for a device/subdevice
          + %dev_name%: your device's name
          + %ip%: the IP entered for your device
          + %test_name%: the name of the test executed
          + %test_result%: the results of the test
          + %snmp_read_community%: the SNMP community for the device
          + %subdev_name%: name of the subdevice
       There are usually several "Dynamic Parameters" for a subdevice;
       if you go to your device, then click parameters next to your
       subdevice, you will get a list of parameters you entered an a
       list of dynamic parameters that NetMRG has created.

III. Installation

   Table of Contents
   6. Software Requirements
   7. Satisfying Requirements
   8. Installation Procedure
   9. Upgrade Procedure
   10. Basic Tutorial
   11. Graph Tutorial
   12. Interface and Disk Cache Tutorial
     _________________________________________________________

Chapter 6. Software Requirements

   NetMRG Requires the following packages for compilation.

     * PHP, Version >= 4.1
     * MySQL, Version >= 3.23
     * RRDTOOL, Version >= 1.0.28
     * Libxml2, Version >= 2.30
     * Libstdc++, Version >= 3.2
       Versions  earlier than this may compile just fine, but may
       experience runtime issues. Specifically, we experienced software
       crashes  on  an SMP machine running version 2.96 that were
       resolved by upgrading to 3.2.
     * NET/UCD-SNMP, Version >= 4.2.2
     _________________________________________________________

Chapter 7. Satisfying Requirements

   Specific instructions to satisfy dependencies for certain platforms
   are presented below. They should be used more as suggestions, not
   necessarily step-by-step instructions. You may need to do more than
   what is listed, or much less, depending on the options you selected
   when installing your operating system.

     * Red Hat Linux
          + In order to satisfy the libstdc++ requirement, Redhat 8.0
            or above is required. NetMRG may not be stable on older
            versions of libstdc++.
          + Install the following RPMs from your Red Hat installation
            media:
               o httpd
               o php
               o php-mysql
               o mysql
               o mysql-server
               o mysql-devel
               o libxml2
               o libxml2-devel
               o net-snmp
               o net-snmp-devel
               o net-snmp-utils
          + Installing RRDTOOL
               o Download an RRDTOOL RPM from http://www.freshrpms.net/
               o OR    Download    the    RRDTOOL   source   from
                 http://www.rrdtool.org/ and compile it.
          + Installing precompiled NetMRG RPM
            Download the latest NetMRG RPM and install it as follows:
$ rpm -Uvh netmrg-0.10.rpm
            You will still have to perform the following tasks listed
            in the Install Instructions:
               o Setup the Database
               o Configure Apache
               o Configure the Installation
               o Setup Crontab
     * FreeBSD (4.8)
          + Install the following ports:
               o www/apache13
               o rrdtool
               o net-snmp
               o libxml2
               o mod_php4 (specify MySQL support)
               o lang/gcc32
          + Run configure, specifying gcc32:
./configure 'LDFLAGS=-L/usr/local/gcc3/lib' 'CPPFLAGS=-I/usr/local/gcc3
/include' 'CXX=g++32'
     _________________________________________________________

Chapter 8. Installation Procedure

    1. Untar  to  some directory like /tmp/ (it will untar into a
       'netmrg' directory)
cd /tmp
wget http://www.netmrg.net/download/release/netmrg.tar.gz
tar xzvf netmrg.tar.gz
    2. Now we need to compile and install the source. cd into the
       directory made by your tar command and type the following:
./configure
make
make install

     Note: If configure fails to locate the needed libraries they can
     be manually specified. Run ./configure --help for details on the
     necessary options.
    3. Setup the database
       Now we need to setup the database. Please change at least the
       password (netmrgpass below).
mysqladmin create netmrg
mysql -u root -p netmrg < share/netmrg.mysql
mysql -u root -p
> grant all on netmrg.* to netmrguser@localhost identified by 'netmrgpa
ss';
    4. You need to modify your apache config in order to make your
       NetMRG installation live (commonly in
       /etc/httpd/conf/httpd.conf).
       The first is to alias /netmrg to the installed web directory
       (commonly /usr/local/var/www/netmrg/webfiles).
Alias /netmrg "/usr/local/var/www/netmrg/webfiles"
       The easier way (if you're running Apache >= 1.3) is to just
       Include the conf file we've created anywhere in your file (near
       the bottom is fine).
Include /usr/local/etc/netmrg.conf
       You  will  need to restart your Apache after making config
       changes. Telling it to reload its config should be good enough
killall -HUP httpd
    5. Configure the Installation
       All of NetMRG's common configuration options are controlled by
       an XML file, normally located at /usr/local/etc/netmrg.xml. You
       should edit this file to make any necessary changes before
       continuing with installation.
       For  most  installations,  you  will at least need to edit
       netmrg.xml to reflect your database username and password.
    6. Setup Permissions
       NetMRG does not need superuser permissions to run, therefore we
       will setup a user for it.
# useradd netmrg
# chown netmrg:netmrg /usr/local/var/log/netmrg
# chown netmrg:netmrg /usr/local/var/lib/netmrg/rrd
    7. Setup crontab
       You can setup the crontab entries in two places: either edit
       /etc/crontab, or run crontab -e as the netmrg user. For 'crontab
       -e', use an entry similar to this:
su netmrg
*/5 * * * * /usr/local/bin/netmrg_cron.sh
exit
       To edit the master crontab, edit /etc/crontab and add a line
       similar to this:
*/5 * * * * netmrg /usr/local/bin/netmrg_cron.sh
    8. Point your web browser at your install, and start graphing
       things! The default login is admin and the default password is
       nimda. Point your web browser at the host you built this on
       (something similar to http://yourhost.com/netmrg/) and please
       change your password!
     _________________________________________________________

Chapter 9. Upgrade Procedure

   After  doing  a  'make install' as described in the chapter on
   Installing NetMRG, the next step is to run the web-based updater.
   This is accomplished by logging into NetMRG with administrator
   privileges, and following the prompts. NetMRG's web interface will
   not  operate  normally and the gatherer will not run until the
   web-based upgrade is performed. Non-administrator users will only
   receive  a  message  directing  them  to  contact their NetMRG
   administrator.

   If you need to perform an upgrade, the updater will prompt you to
   apply all updates. Clicking this link will perform all necessary
   updates for this release.

   If for some reason an update fails, or you would like to re-apply an
   update for any reason, you can goto the updater and goto View all
   available updates. This will present you a list of the updates that
   are available and their status (applied, not applied, or error). If
   you have problems updating from one version to the next, you can
   force  individual updates that fail so that they're considered
   'applied'. Once all updates are applied for a particular version,
   the database will be automatically updated to the latest version
   that has all updates applied.
     _________________________________________________________

Chapter 10. Basic Tutorial

   This brief tutorial will walk you through the major features of
   NetMRG. The tutorial assumes that you have installed the Net-SNMP
   daemon on a system which you want to monitor.

    1. Create a new group.
         a. Click on Groups in the Monitoring menu.
         b. Click on Add.
         c. In the Name textbox, enter a name for your group, such as
            "Home."
         d. In the Comment textbox, enter a description for your group,
            such as "Computers in my house."
         e. In the Parent drop down box, select -Root- as we want this
            device in the root of the device tree. You can create
            nested groups.
         f. Click the Save Changes button to create your new group.
    2. Create a new device.
         a. Click on the name of your new group. You are now looking at
            an (empty) list of devices in your new group.
         b. Click on Add in the Monitored Devices section.
         c. Click on Create a new device.
         d. In the Name textbox, enter a name for your device, such as
            "My Server."
         e. In the IP or Host Name textbox, enter a domain name or IP
            address of your device. Domain names must be resolvable
            from your monitoring host.
         f. In  the SNMP Support dropdown, select the appropriate
            version. If unsure, select SNMPv1.
         g. In  the  SNMP  Read Community textbox, enter the SNMP
            community string for your device.
         h. Click the Save Changes button to create your new device.
    3. Create a new sub-device.
         a. Click on the name of your new device. You are now looking
            at an (empty) list of sub-devices in your new device.
         b. Click on Add.
         c. In  the  Name textbox, enter "System." This will be a
            pseudo-sub-device used for monitoring the device as a
            whole.
         d. In the Type drop down box, ensure that Group is selected.
         e. Click the Save Changes button to create your new "System"
            sub-device.
    4. Add graph templates.
       Since devices running Net-SNMP are common, NetMRG comes with
       several graph templates for commonly used variables.
         a. Click on Template Graphs in the Graphing menu.
         b. Click on the Apply Template To... icon (hover to see the
            caption) next to the Linux Box - CPU Utilization template
            graph.
         c. In the Subdevice drop down box, select My Server - System.
         d. Click on the Save Changes button to apply the template.
         e. Repeat this step for any other graph templates you find
            interesting.
    5. View your new graphs.
         a. Click on Device Tree in the Reporting menu.
         b. Click on the name of your group to expand it.
         c. Click on the graph icon beside your device (orange graphs
            mean there are graphs associated with this device, black
            means there are no graphs attached to this device.
         d. You should now see graphs! If you don't, you may need to
            wait for the gatherer to perform another cycle.
     _________________________________________________________

Chapter 11. Graph Tutorial

   This tutorial aims to show you the basics of how to create a Custom
   or Template Graph. It will be useful to look at existing graphs that
   are supplied with NetMRG for examples.

   Custom and Template graphs are very similar, but have slightly
   different applications. A Custom Graph is designed to be used only
   once, while a Template Graph is designed to be applied to multiple
   devices and subdevices

   Let's try a basic graph using no subdevice parameters.

    1. Creating a Template/Custom Graph
         a. Goto the Template Graph item under the Graphing menu.
         b. Click on Add.
         c. Click on Add.
            This leads you to a screen where you can combine multiple
            monitors (usually of the same subdevice) into one graph
            using various styles; areas, stacked areas, and thicknesses
            of lines.
         d. Click on add
            This  page allows you to edit various aspects of this
            monitor's appearance.
            Name:  Name  that will be displayed in the legend. If
            omitted, this item will not be displayed in the legend.
            Item Type: can be various thickness of lines (1 being
            thinnest, 3 being thickest), area (which will always start
            at '0' from the graph - positive or negative), and stack
            (which will 'stack' on top of other areas and stacks).
            Item Color: color of the item on the graph and in the
            legend. It can be omitted to have a transparent item.
            Fixed Value or Multiplier: change the value displayed by
            your monitor.
            The rest of the checkboxes should be self explanatory
         e. By repeating these steps you can add multiple monitors to a
            single graph.
     _________________________________________________________

Chapter 12. Interface and Disk Cache Tutorial

   The Interface and Disk Cache (that is available next to your device
   names  under  the  Groups menu) are quick tools for setting up
   interfaces and disk graphs on your devices.

   These  caches provide useful mechanisms for indexing disks and
   interfaces. For example, commonly you want to graph your interfaces
   by their ifName (interface name) which is commonly something like
   'Se2/1'. Doing a lookup each time you gather data would be very
   time-consuming, so the cache allows you monitor an interface by
   using it's name, and NetMRG will do the mapping to the correct
   ifIndex. NetMRG will do the same thing for disks

     * The options from next to your device are as follows
         1. View Interface Cache: This is only available after you have
            cached your interfaces. Most of the management is done
            through here
         2. Recache Interfaces: If you add or remove interfaces to your
            device, this can force a recache. You can also setup NetMRG
            to automatically recache your interfaces when the device
            reboots or the number of interfaces changes by editing your
            device.
         3. View Disk Cache: Like the interface cache; the list of
            disks that NetMRG currently knows about.
         4. Recache Disks: You'll probably only do this once when you
            create your device (unless you add/remove disks a lot).
     * Inside the Interface Cache
         1. Checkboxes: You can select multiple interfaces to monitor
            by using the '*' (all), '0' (none), and '-' (inverse)
            items, or by clicking each checkbox individually.
         2. View: View the graph that has been created by clicking
            'monitor/graph'.
         3. Monitors: View the monitors under the subdevice that was
            created by clicking 'monitor/graph'.
         4. Monitor/Graph: Apply the graph template selected in the
            dropdown box by clicking this item
         5. Dropdown  box:  The  list of graph templates that are
            available in your NetMRG installation; it's very useful
            when you have different types of interfaces with different
            monitors you want applied to an interface (maybe you have
            one graph with a maximum line, or different multiplers to
            show bits instead of bytes).
     * Inside the Disk Cache
         1. The options inside the Disk Cache are very similar to the
            Interface Cache. Refer there for their descriptions.

IV. Feature Guide

   Table of Contents
   13. External Authentication
   14. Workday Highlighting
     _________________________________________________________

Chapter 13. External Authentication

   NetMRG  handles authentication internally by default, checking
   passwords against its own database. It is possible for NetMRG to use
   other  systems  for  username/password authentication. This is
   accomplished by handing off the authentication task to the HTTP
   server.

     Note: You must still create accounts in NetMRG for each user when
     using external authentication in order to assign them permissions
     and rights.

   Details on methods of authentication using the Apache web server are
   available here. There are add-on modules for Apache that allow for
   various  authentication  mechanisms, including LDAP and custom
   database authentication systems. Using Apache-based authentication,
   it is possible for an organization to maintain users and passwords
   in one location for a variety of web-based services, only needing to
   set up application-specific permissions in each application.

   To  enable  external  authentication,  edit netmrg.xml and set
   externalAuth to true.

   There  is  also  a  way  to  map all users that are able to be
   authenticated to a single user's permissions. This is accomplished
   by  creating  a user named NETMRG_DEFAULT_MAP and assigning it
   whatever permsissions you see fit (SVO, Read Only, etc). This is
   often useful when you have a large staff that is authenticated
   centrally  and you would like to allow them all to view graphs
   without having to create an account for each employee.
     _________________________________________________________

Chapter 14. Workday Highlighting

   Shading specific time periods on a graph is possible using NetMRG.
   Typically  this  is  used  for highlighting the workday or SLA
   distinctions based on time of day. The following steps will walk you
   through the setup of workday highlighting on a graph.

    1. Add a new Graph Item.
    2. Select Show Advanced.
    3. Leave the title of the item blank if you want it to be omitted
       on the legend, otherwise enter a label such as "Business Hours."
    4. Set the Item Type to Area.
    5. Select a color for the highlighting.
    6. Select Fixed Value in the Monitor drop-down.
    7. Type in "INF" as the fixed value. This allows the graph item to
       take on an infinite height, which is rendered by filling the
       background.
    8. Unselect all checkboxes under Legend.
    9. Enter start and stop times in their respective text boxes. Times
       should be prefixed with a plus sign ('+') and be consistent with
       GNU Date Input Formats. An example would be +9am for a start
       time  and +5pm for an end time, to highlight the "typical"
       business day.
   10. Click Save Changes and observe the colored rectangle cover a lot
       of the data on your graph.
   11. Click Move Up beside your new graph item until it reaches the
       top of the list. You now have a graph with workday highlighting!

V. Reference

   Table of Contents
   I. Manual Pages
   II. Web Interface Descriptions

I. Manual Pages

   Table of Contents
   netmrg-gatherer -- Gathers data from devices.

netmrg-gatherer

Name

   netmrg-gatherer -- Gathers data from devices.

Synopsis

   netmrg-gatherer [OPTIONS]

Description

   netmrg-gatherer performs tests on all enabled hosts in NetMRG. It
   will store results in RRD files using RRDTOOL(1) and perform actions
   in response to events.

Global Options

   -h

          Displays usage information and exits.

   -v

          Displays software version and exits.

Operating Options

   -t num threads

          Run  no  more  than  the  specified  number  of threads
          simultaneously.

   -X

          Become a daemon.

   -M mode

          Mode of operation. This option controls the manner in which
          the gatherer operates. The modes currently supported are
          "once" (the default, best used manually or via cron) which
          runs all tests once then exits, and "wait" (best used when
          daemonized) which will run all tests, wait for the gatherer
          interval to expire, and then repeat.

     Note: To run netmrg-gatherer daemonized, the recommended set of
     options at this point is "-X -S -M wait" which will daemonize,
     repeat the gather cycle on each interval, and write output to
     syslog.

Logging, Debugging and Verbosity Options

   -S

          Syslog. Logs messages to syslog instead of stdout.

   -b

          Bland. Disables color output. By default, netmrg-gatherer
          will display color output when connected to a terminal, but
          will use bland output when its output is written to a file.
          This  option  should  only be needed when the detection
          mechanism has failed, or the user doesn't want to see colors
          when running the program by hand.

   -a

          All. Displays all log messages.

   -m

          Most. Displays more messages than default.

   -q

          Quiet. Silences all output.

   -l level mask

          Only outputs messages that match the specified level mask.

   -c component mask

          Only outputs messages that match the specified component
          mask.

   -s

          Safe.  Tries  to  keep sensitive information out of the
          displayed messages.

Configuration File Options

   -C config file

          Uses the specified XML configuration file.

   -K config file

          Parses the specified XML configuration file and exits.

Database Options

   -H host

          Uses the specified host name for connecting to the database.

   -D database

          Uses the specified database name.

   -u user name

          Uses the specified user name for connecting to the database.

   -p [password]

          Uses the specified password for connecting to the database.
          If no password is specified, the user is prompted to enter
          one.

Recaching Options

   -i device id

          Performs an interface recache on the specified device.

   -d device id

          Performs a disk recache on the specified device.

     Note: Recaching options are intended for use by the web interface.
     These options should not be generally used by end users.

II. Web Interface Descriptions

   Table of Contents
   Monitoring -- Items associated with setting up gathering of
          information

   Groups -- A container for a collection of devices
   Devices -- A container for a collection of sub-devices
   Sub Devices -- A container for a collection of monitors
   Monitors -- A container for a collection of tracked values
   Events -- A container for a collection of actions to take based on
          monitor values

   Conditions -- A collection of paramenters to trigger an Event
   Reporting -- Items whose purpose is to display and report
          information to the user.

   Graphing -- Where graphs (compositions of one or more monitors) are
          created.

   Tests -- Tests represent actions taken to gather data.
   Notifications -- Configuration for notifying users when events
          occur.

   Admin -- Administrative functions not related to the reporting or
          gathering of information.

   Help -- How to get assistance with NetMRG.

Monitoring

Name

   Monitoring -- Items  associated  with  setting up gathering of
   information

Description

Groups

   Management of Groups, Devices, Sub-devices, Monitors, Events, and
   the Notifications used by the events.

Device Types

   Types of devices; unused at this point.

Notifications

   Types of notifications that can be tied to events.

     * Name - Descriptive name of the notification
     * Command - Command to execute for the notification. Notifications
       that start with a '/' will use the ful path the the specified
       command, whereas commands without a starting '/' will start
       using NetMRG's libexec directory. Parameter substitution is
       fully functional here, as well.

Groups

Name

   Groups -- A container for a collection of devices

Description

List

   List of all groups. This is the default view when you first click on
   Groups in the nav bar.

     * Name  - click on the name of the group to view the devices
       contained in it
     * Comments - description of a group
     * Add - add another group
     * View - views graphs assocated with this group and allows you to
       edit what graphs are associated with the group
     * Edit - edits parameters of a group
     * Delete - deletes the group and all devices, sub-devices, etc
       underneath it

Edit

   Edit information about a group

     * Name - Name of the group
     * Comment - Description of the group
     * Parent - Parent group for this group; sets up subgroups

Devices

Name

   Devices -- A container for a collection of sub-devices

Description

List

   List of devices in a group; this is the default view when you click
   on a group name.

     * Name - click on the name of the device to view the sub devices
       contained in it
     * SNMP Options
          + View Interface Cache - views a list of cached interfaces
            (interfaces and values that NetMRG knows about)
          + Recache Interfaces - recache the interface list (to get new
            ifAliases or new interfaces, for example)
          + View Disk Cache - views a list of cached disks (disks and
            their values that NetMRG knows about)
          + Recache Disks - recache the list of disks (to get new disks
            or paths, for example)
     * Add - add a new device or an existing device to this group
     * View - views graphs assocated with this device and allows you to
       edit what graphs are associated with the device
     * Edit - edits parameters of a device
     * Delete - deletes the device all sub-devices, monitors, etc
       underneath it

Add

     * Create a new device - creates a new device in the current group
     * Add  an  existing device to this group - puts a link to an
       existing device in this group (does not copy it, just refers to
       it; they are the same device
          + Device - Name of the device to add to this group

Edit

   Edit info about a device

     * General
          + Name - Name of a device; commonly the hostname
          + IP or Hostname - IP address of the device, or hostname
            (must be resolvable by the monitoring box)
          + Device Type - Type of device (currently not used)
          + Disabled - Don't gather data for this device
     * SNMP
          + SNMP Support - The version of SNMP to use when querying
            this device, if any.
          + SNMP Read Community - The SNMP community string to use when
            querying this device.
          + Recaching  Method  - Methods used by the gatherer for
            determining if its SNMP cache is stale.
               o Never  refresh  cache  - The gatherer will never
                 automatically refresh the cache. The cache must be
                 refreshed manually when changes occur.
               o Refresh on SNMP agent restart - The gatherer will
                 refresh  the cache if the SNMP agent uptime goes
                 backwards, or if the device has become reachable again
                 after a period of unreachability. This option is
                 useful for devices that change interface counts too
                 often, such as a router serving DSL sessions.
               o Refresh on interface count change - The gatherer will
                 refresh the cache if the "ifNumber" value has changed
                 since the last gathering. This typically happens when
                 cards are added to or removed from a router, or when
                 PPP sessions go up or down. (Also includes the above.)
               o Refresh on interface count mismatch - The gatherer
                 will refresh the cache when the number of items in the
                 interface cache do not match the "ifNumber" on a
                 device. This can prevent problems when devices are in
                 the process of booting. This is the preferred option
                 for most devices. (Also includes the above.)
               o Always refresh cache - Always refresh the cache on
                 each gather. This is useful for devices that are
                 always changing, when you care about the changes.
     * Advanced SNMP Options
          + Disable SNMP Uptime Check - Allows NetMRG to work with
            devices  that  are broken or abnormal with respect to
            system.sysUpTime.0. NetMRG usually uses this value to
            determine  if  a host is up and if the SNMP agent has
            restarted since the last polling cycle. It should only be
            used when absolutely necessary as it reduces the robustness
            of the SNMP avoidance and recaching mechanisms which can be
            crutial to performance.
          + SNMP UDP Port - UDP port to direct SNMP queries to (default
            is 161).
          + SNMP Timeout (microseconds) - Number of microseconds to
            wait for a response to a query before retrying (default is
            1000000 or 1 second). An exponential backoff is used after
            each failed query; this is the initial delay.
          + SNMP Retries - Number of times to query the device before
            giving up (default is 3).

SNMP Cache View

   List of cached values for a device

Interface Cache

   List of Interfaces NetMRG knows about

     * Index - SNMP ifIndex
     * Status - SNMP ifAdminStatus/ifOperStatus
     * Name - SNMP ifName
     * Description - SNMP ifDescr
     * Alias  - SNMP ifAlias (commonly the 'description' field on
       routers and switches)
     * IP Address - IP address of this interface
     * MAC Address - MAC address of this interface
     * View - View the graph for this interface (if monitored)
     * Monitors - View the monitors for this interface (if monitored)
     * Monitor/Graph  -  Choose  to graph this interface (adds an
       interface  subdevice  based  on ifDescr, adds monitors for
       ifInOctets, ifOutOctets, and ifInErrors for the subdevice, and
       assocates a graph with the device)

Disk Cache

   List of Disks NetMRG knows about

     * Index - SNMP dskIndex
     * Device - SNMP dskDevice
     * Path - SNMP dskPath
     * View - View the graph for this disk (if monitored)
     * Monitors - View the monitors for this disk (if monitored)
     * Monitor/Graph - Choose to graph this disk (adds a disk subdevice
       based on dskPath, adds monitors for KB Used and KB Available for
       the subdevice, and assocates a graph with the device)

Sub Devices

Name

   Sub Devices -- A container for a collection of monitors

Description

List

   list of sub devices in a device

     * Sub Devices - Name of the sub devices for this device; click on
       a name to view its monitors
     * Type - Type of sub device (group, interface, or disk)
     * Add - add a new sub device to this device
     * Parameters - View/edit available parameters for this sub device
       (such as what ifName to use to graph this interface)
     * View - views graphs assocated with this sub device and allows
       you to edit what graphs are associated with the sub device
     * Edit - edits options of a sub device
     * Delete  - deletes the sub device all monitors, events, etc
       underneath it

Edit

   edit info about a sub device

     * Name - Name of this sub device
     * Sub-Device Type - Type of sub device (group, interface, or disk)

Parameters

   items that alter what interface or disk a sub device monitors

Configured Parameters

     * Name - Name of paramter to match this sub device to (commonly
       'dskPath' for disks or 'ifDescr' for interfaces)
     * Value - Value of the Name to match (commonly '/' for disks or
       'eth0' for interfaces)
     * Add - Adds a Name/Value pair for this sub device
     * Edit - Edits the values above for this sub device
     * Delete - Deletes this Name/Value pair for this sub device

Dynamic Parameters

     * Name - Name of paramter that is available in template graphs as
       a wildcard (put between two '%' in the title)
     * Value - Value of the Name that will be displayed

Monitors

Name

   Monitors -- A container for a collection of tracked values

Description

List

   list of monitors for a sub-device

     * Test - Name of what is being monitored; click to view/edit
       events for this monitor
     * Data - Information about the last gather of this monitor
          + Value - value gathered in the last run
          + Rate of Change - Difference between the current value and
            the last value, divided by the time between samples
          + Time Stamp - Date/Time of last gather for this monitor
     * Graph - A small representation of the current data gathered for
       this monitor; click on the graph for more options
     * Add - add a new monitor to this sub device
     * Edit - edits options of a monitor
     * Delete - deletes the monitor and everything underneath it

Edit

   edit info about a monitor

     * Monitoring Type - specifies the type of test used for this
       monitor (Script, SNMP, SQL, or Internal)
     * Test Options - this section varies depending on the setting of
       Monitoring Type
          + Script Options
               o Script Test - specifies which script to run
               o Parameters - specifies a string which will be appended
                 to the script's parameters. Parameter expansions are
                 honored in this field.
          + SNMP Options
               o SNMP Test - specifies which SNMP test to run
               o Parameters - specifies a string which will replace the
                 expansion %parameters% in SNMP OIDs.
          + SQL Options
               o SQL Test - specifies which SQL test to run
               o Parameters - specifies a string which will replace the
                 expansion %parameters% in SQL queries.
          + Internal Options
               o Internal Test - specifies which internal test to run
               o Parameters - specifies a string used by the internal
                 test
     * Graphing Options
          + Data Type - the RRDTOOL data type for this monitor
               o Absolute - a counter reset when read
               o Counter - an incrementing value interpreted as a rate
                 (such as calculating your average speed with odometer
                 readings)
               o Derive  - a counter without overflow protection,
                 permits graphing of negative rates
               o Gauge - the value read is the value used, such as
                 temperatures
            For a more complete discussion on when to use which data
            type, see the RRD Create manual page.
          + Minimum Value - the minimum value accepted by RRDTOOL as
            valid.
          + Maximum Value - the maximum value accepted by RRDTOOL as
            valid.

Events

Name

   Events -- A container for a collection of actions to take based on
   monitor values

Description

List

   list of events for a monitor

     Note: Events displayed in bold are currently triggered.

Edit

   edit info about an event

     * Name - a name for the event, unique within a device. This name
       is used when displaying entries in the Event Log.
     * Trigger  Type  - when to trigger an event. When events are
       triggered, they are logged in the Event Log and any Responses
       associated with the event are executed.
          + On Change - The event is triggered when its conditions are
            met, and the conditions had not been met on the last run.
            Example: If you have an event to warn you when CPU Usage
            climbs above 80%, it will trigger this condition if the
            previous value was 72 and the current value 85, but not if
            the previous value was 82 and the current value 85. This
            prevents duplicate triggerings to a situation spanning
            multiple polling intervals.
          + Never (disabled) - This event is ignored and responses for
            it are not processed.
     * Situation - the severity of the situation the event indicates.
          + Disabled - the tested service is in a non-operational, yet
            normal, condition.
          + Normal - the tested service is operating normally.
          + Warning - the tested service is operating in a degraded
            condition.
          + Critical - the tested service is not operating acceptably.

     Note: Situations for an event propagate up the Device Tree. Each
     level  takes on the worst situation of its descendents. For
     example, if you have a Critical situation, the Event as well as
     the Monitor, Sub-Device, Device, and Groups above that Event will
     take on the situation Critical as well.

Conditions

Name

   Conditions -- A collection of paramenters to trigger an Event

Description

List

   list of conditions for an event

Edit

   edit info about an condition

     * Logical Operation - the logical operator (AND or OR) to apply
       between this condition and the previous one. This parameter is
       not valid, nor is displayed, for the first condition in a set of
       conditions.
     * Value Type - the type of data about the Monitor to be used for
       comparision.
          + Current Value - the current value of the Monitor
          + Delta Value - the difference between the current value and
            the previous value.
          + Rate of Change - the delta value divided by the difference
            in timestamps between the current and previous values. When
            using counters, this gives you the "per second" value.
     * Condition - the arithmetic operator to use when comparing the
       Value Type of the Monitor and the Value of the Condition.
     * Value - the number used for comparision.

Reporting

Name

   Reporting -- Items  whose  purpose  is  to  display and report
   information to the user.

Description

Device Tree

   Displays  graphs attached to Groups, Devices, etc, down to the
   monitor level. Graph icons next to each level represent where views
   are attached to items. Orange icons mean one or more graphs are
   linked here; grey icons mean there are non currently.

   Groups will have a Slide Show icon next to them which will show all
   views in subgroups and devices underneath this group.

   Most items will also have a pencil icon (when a user has Read/Write
   permissions) in order to quickly edit this item.

Event Log

   When events are triggered, the record of their status changes can be
   viewed here.

Slide Show

   Starts an in-order or random slide show of all views. A 'Resume
   Slide Show' link will also appear to go back to the place where you
   left  off if you leave the slide show. If you have auto-scroll
   enabled, clicking anywhere on the page will pause the slideshow, and
   clicking again will resume scrolling.

Graphing

Name

   Graphing -- Where graphs (compositions of one or more monitors) are
   created.

Description

Custom Graphs

   Custom Graphs are graphs meant to only be used once. If you have an
   application where you are going to apply the same type of monitors
   to more than one device (even if the parameters are different), use
   a template graph.

Template Graphs

   Graphs intended to be used more than once. Template graphs are
   applied to a sub-device at the view level. It will look for the same
   type of monitors as it was created with and use the matching monitor
   types from whichever sub-device was selected.

   Template graphs also have the added benefit of being applied to a
   sub-device, allowing you to automatically create the same monitor
   types inside a new sub-device, quickly adding more monitors to your
   device.

Tests

Name

   Tests -- Tests represent actions taken to gather data.

Overview

   A monitor can be thought of as a specific instantiation of a test.
   There  are  currently four types of tests supported by NetMRG:
   Scripts, SNMP, SQL, and Internal.

Scripts

   Arbitrary scripts can be executed and their output or return values
   used as values to be monitored.

Edit

     * Name - An description used to identify the test.
     * Command - The program to execute for this test. If the value of
       this field does not start with a slash, it is assumed that the
       command  is located in the default NetMRG tests directory.
       Commands are passed to a shell for execution by the gatherer.
     * Data Type - The type of data gathered from the program.
          + Standard Out - Data from the program's standard output is
            returned. The program should output data on one line which
            should be a numeric value or "U" to specify an unknown
            value.
          + Error Code - The integer return code from the executed
            program is returned.

SNMP

   SNMP  (Simple  Network Management Protocol) is widely used for
   accessing statistics on network infrastructure and hosts.

Edit

     * Name - An description used to identify the test.
     * SNMP OID - The OID to be obtained. The value may be numeric or
       of any form your Net-SNMP library can understand. Because the
       installed MIBs can vary from machine to machine, use of numeric
       (or at least common) values is preferred.
     * Type - specifies whether to GET or WALK the OID
          + Direct (Get) - The specific OID specified will be returned.
          + Nth Item (Walk) - The OID specified will be walked until
            the Nth item is encountered. This item will be returned.
     * Item # - Specifies which item to return when doing an SNMP walk.
       This option has no effect on SNMP GETs.

SQL

   SQL (more specifically, MySQL) tests are useful when you have data
   that is otherwise in-accessable, or takes a long time to gather.
   Rather than making the netmrg-gatherer run a script test that will
   take a long time, consider writing the script to run periodically
   (out of Cron, for example) and dumping its output into a MySQL
   database for NetMRG to gather later. This greatly reduces NetMRG's
   runtime.

Edit

     * Name - An description used to identify the test.
     * Host - The host running the MySQL server you wish to connect to.
     * User - The MySQL user to connect as.
     * Password - The password for the MySQL user.
     * Query - The SQL query to perform.

     Note: No default database is specified, so be sure to include the
     database name in the table names that you use in your query.
     * Column Number - The column to return. NetMRG will collect the
       value from this column in the first row of the result set.

Internal

   Internal tests are compiled into the gatherer. These tests typically
   start as script tests, and are then migrated to the gatherer for
   efficiency. At the moment, the gatherer source code is the only
   documentation  for  such  tests, however some are obvious. The
   intention is to expand this category to include DSO-based test
   packages.

Notifications

Name

   Notifications -- Configuration for notifying users when events
   occur.

Description

Edit

     * Name - A name used to identifiy the notification.
     * Command - A shell-interpreted command to run to notify the user
       of an event. Parameter expansions will be made to this field as
       outlined below.
     * Disabled - When selected, the specified will not be run. Useful
       for temporary suspension of notifications.

Variable Substitutions

   The following parameters will be substituted in the command field:

     * %dev_name% - The device's name
     * %ip% - The IP or host name for the device
     * %event_name% - The name of the triggered event causing the
       notification.
     * %situation% - The situation of the triggered event (Normal,
       Warning, etc.)
     * %current_value% - The monitor's last gathered value.
     * %delta_value% - The change between the monitor's last gathered
       value and the previous.
     * %rate_value% - The rate-of-change based on the last two gathered
       values and the time difference between them.
     * %last_value% - The value from the gathering period before last.

Admin

Name

   Admin -- Administrative functions not related to the reporting or
   gathering of information.

Description

Users

   The area where you manage users that are allowed to view or modify
   graphs and parameters in NetMRG. If you're using External Auth, you
   will also need to come here to create users to setup permissions for
   them to access your NetMRG installation.

Prefs

   Preferences for your users (such as auto-scroll in the Slide Show,
   or changing your password). If you're an admin, you will be able to
   change your users' prefs here, or you will only be able to change
   your own.

Logout

   Logout of NetMRG. NetMRG will keep its session active for 24 hours;
   this may allow other users to come and use your computer to view or
   modify your graphs if you don't close your browser. Logging out of
   NetMRG will prevent accidental (or malevolent) activities in your
   NetMRG installation.

Help

Name

   Help -- How to get assistance with NetMRG.

Description

About

   Some basic information about NetMRG, its authors, the license, and
   credits to noteworthy contributors.

Manual

   A link to the NetMRG Manual.

Forum

   A link to the best place to go for useability questions.

Bugs

   If you think you have found a bug, submit it here.
     _________________________________________________________

Chapter 15. Contact Information

   We have great aspirations for what NetMRG could become, but as with
   all open-source software, such aspriations can only be achieved with
   the active participation of the open-source community. Please do
   your part and support software that you use by helping to answer
   support questions on forums and report bugs to developers. Thanks!

     * Main Website - http://www.netmrg.net/
     * Support Forum - http://lists.netmrg.net/
     * Bug Reporting and Tracking - http://bugs.netmrg.net/
