Torrus Installation Instructions

Note to Debian users: if you have installed torrus using Debian packages, all the needed dependencies outlined below have been satisfied and installation tasks carried out by the Debian package management system. The information below is provided for information purposes only and may not apply to the Debian torrus installations. Please see the README.Debian file in /usr/share/doc/torrus-common for more details.

Required Software

Quick Installation with SDBS

# This works in Debian or Ubuntu.
# You need to install corresponding packages in orther systems
apt-get update && apt-get install -y git gcc g++ make automake \
  libtool apache2 libapache2-mod-fcgid pkg-config xz-utils bzip2 \
  redis-server redis-tools

mkdir -p /opt/pm/src
cd /opt/pm/src

git clone https://github.com/oetiker/sdbs.git
cd sdbs
PREFIX=/opt/pm/thirdparty ./build_perl5-current.sh
PREFIX=/opt/pm/thirdparty ./build_rrdtool-current.sh

echo 'PATH=/opt/pm/thirdparty/bin:/opt/pm/bin:$PATH; export PATH' \
  >/etc/profile.d/torrus_path.sh

logout
#login again

# Watch out the group name, your OS may have a different group name for
# Apache (this here works for Debian or Ubuntu)
useradd -rU torrus
usermod  -aG torrus www-data

cd /opt/pm/src/
git clone https://github.com/ssinyagin/torrus-newfeatures.git
cd torrus-newfeatures/src
sh setup_tools/cpanm_install_modules.sh
autoreconf
./configure --prefix=/opt/pm piddir=/var/torrus/run
make install

# systemd unit files
cp systemd/torrus.service /lib/systemd/system/
cp systemd/torrus.tmpfile /usr/lib/tmpfiles.d/torrus.conf
systemctl enable torrus

# install plugins as necessary
cd ../plugins/cbqos/
autoreconf
torrus install_plugin .

mkdir /opt/pm/share/www
vi /opt/pm/share/www/index.html
=== cut ===
<html>
<head>
<title>redirect page</title>
<META http-equiv="Refresh" CONTENT="0; URL=/torrus/">
</head>
<body></body>
</html>
=== cut ===

vi >/etc/apache2/sites-available/200-torrus.domain.com.conf
=== cut ===
<VirtualHost *:80>
 DocumentRoot "/opt/pm/share/www"
 ServerName torrus.domain.com
 ScriptAlias /torrus "/opt/pm/torrus/bin/torrus.fcgi"
 FcgidMaxProcessesPerClass 10
 FcgidMaxRequestsPerProcess 100
 <Location />
   Options Indexes
   Require all granted
 </Location>
 <Location /torrus>
    Options       +ExecCGI
 </Location>
</VirtualHost>
=== cut ===

a2ensite 200-torrus.domain.com
systemctl restart apache2

mkdir -p /srv/torrus/collector_rrd
chown torrus:torrus /srv/torrus/collector_rrd
chmod g+s /srv/torrus/collector_rrd

Installation Procedure (in detail)

Create the group torrus and a user torrus. The user should be a member of this group.

Add your Apache daemon user to the group torrus. Other users that will run any of Torrus processes must be included in this group.

For example, in Solaris (you may need to specify the GID and UID numbers according to your local administration policy):

groupadd torrus
useradd -c 'Torrus Daemon' -d /usr/local/torrus -g torrus torrus
usermod -G www,torrus www

Further installation process is mostly as usual (watch out piddir setting, as described above):

./configure
make
make install

The default directory layout is described below. Should you need to change it, there is a number of variables and configuration options that you may use as ./configure arguments. See ./configure --help for details. Alternatively you can utilize the script ./setup_tools/configure_fhs, which is designed to provide an FHS compliant setup. The script is equivalent to executing

./configure \
 --prefix=/opt \
 --mandir=/opt/share/man \
 pkghome=/opt/torrus \
 sitedir=/etc/opt/torrus

Do not try to change any paths by supplying them as make variables, this would most probably break the installation. The only make variable that is supported is DESTDIR. It may be used for preparing the package for further distribution. For example, the following command would install all Torrus files as if /tmp/stage were the root of the filesystem:

make DESTDIR=/tmp/stage install

The presence of prerequisite Perl modules is checked during the execution of ./configure. You can disable this by giving --enable-pkgonly option.

The installer sets up the site configuration files, but only if they don't already exist.

Plugin modules should be installed separately, after the Torrus software is successfully installed. For each plugin, unpack the plugin distribution archive to some directory, and execute

torrus install_plugin <UNPACKED_PLUGIN_DIR>

A number of directories is set up by default under the path /var, and they must be writable by all Torrus processes, including the Apache web server.

You can control these directories' access rights by setting the following environment variables: var_user, var_group, var_mode, like follows:

./configure var_group=wwwrun

Default values for operating systems other than Cygwin are: var_user=torrus, var_group=torrus, var_mode=775. Setgid bit is set by default for these directories.

Attention for the systems that have /var/run in a temporary directory. By default, Torrus installer creates a directory /var/run/torrus to store the daemon PID files. After the server reboot, the temporary filesystems are cleaned up, and the directory would not exist any more. Use piddir=/var/torrus/run or similar in the ./configure arguments.

Logging

The collector and monitor daemons send their logs to a local SYSLOG server in "local0" facility by default.

The SYSLOG facility can be altered in torrus-siteconfig.pl:

$Torrus::Log::syslogFacility = 'local5';

Also the following two statements would revert to the old behavior (log files written directly by the daemons). Please not that the daemons would not rotate the logs on SIGHUP:

$Torrus::Collector::useSyslog = 0;
$Torrus::Monitor::useSyslog = 0;

If your system policy does not allow using the system SYSLOG daemon, you can run your own (rsyslogd is recommended) and attach it to a UNIX socket. In this case, the following statement in siteconfig would direct all logging to an alternative SYSLOG socket:

$Torrus::Log::syslogSockOpt = ['unix', '/home/jsmith/tmp/syslog.sock'];

Torrus directory layout

/usr/share/
      Home directory for Torrus distribution files

/usr/share/torrus/conf_defaults/
      torrus-config.pl and other configuration files

/usr/share/torrus/bin/
      Command-line executables

@docdir@/
      POD and TXT documentation files

/usr/share/doc/torrus-common/examples/
      Miscelaneous example files

/usr/share/perl5/
      Perl libraries

/usr/share/torrus/plugins/
      Plugins configuration

/usr/share/torrus/scripts/
      Scripts

/usr/share/torrus/sup/
      Supplementary files, DTDs, MIBs, color schemas, web plain files

/usr/share/torrus/templates/
      Renderer output templates

/usr/share/torrus/xmlconfig/
      Distrubution XML files

/etc/torrus/
      Site configurable files

/etc/torrus/conf/
      Place for torrus-siteconfig.pl and other siteconfigs

/etc/torrus/conf/discovery/
      Devdiscover input files

/etc/torrus/templates/
      User-defined Renderer output templates

/etc/torrus/xmlconfig/
      User XML configuration files

/usr/share/man
      Place for man pages. All articles have the prefix C<torrus_>

/var/log/torrus/
      Daemon logfiles

/var/run/torrus
      Daemon PID files

/var/lib/torrus/session_data/lock
/var/lib/torrus/session_data/store
      Web interface session files

/var/lib/torrus/collector_rrd
      Recommended directory for RRD files generated by collectors

Configuring Torrus

The datasources are configured with %Torrus::Global::treeConfig hash in torrus-siteconfig.pl.

In this hash, the keys give the tree names. The values for each tree name are pointers to hashes, with the following keys and values: xmlfiles points to an array of source XML file names; run points to a hash with the names of the daemons that would be automatically launched for the tree; desription gives a short line describing the tree contents.

Two additional arrays: @Torrus::Global::xmlAlwaysIncludeFirst and @Torrus::Global::xmlAlwaysIncludeLast give a list of source XML files that are included in every tree, in the beginning or in the end of the XML files list. By default, this array consists of two files: defaults.xml and site-global.xml. The second one is resided in the user-configurable directory, and you can use it to override any default settings.

Example:

@Torrus::Global::xmlAlwaysIncludeFirst =
    ('defaults.xml', 'site-global.xml');

%Torrus::Global::treeConfig = (
  'tree_A' => {
    'description' => 'The First Tree',
    'xmlfiles' => [qw(a1.xml a2.xml a3.xml)],
    'run' => { 'collector' => 1, 'monitor' => 1 } },
  'tree_B' => {
    'description' => 'The Second Tree',
    'xmlfiles' => ['b1.xml', 'b2.xml'],
    'run' => {} }
 );

XML files are read additively within each tree, in the order as they are listed. The XML compiler searches for the files in several directories, listed in @Torrus::Global::xmlDirs. By default, this list conssists of two paths, one for Torrus distribution files, and the other for user files.

Files generated by devdiscover usually contain include statements which add the vendor-specific XML files to the compilation.

Below follows a short description of some XML files that come with Torrus distribution. Only site-global.xml is installed in the directory for user-configurable files, all others are placed in the distribution directory.

In addition, the distribution package contains several example files.

For more details about XML configuration, see Torrus User Guide and Torrus XML Configuration Guide.

Site configuration options

In addition to %Torrus::Global::treeConfig, you may wish to set some other parameters in your site configuration file (torrus-siteconfig.pl).

See torrus-config.pl for the complete list of varaibes that you may override in your site config. Among them, most interesting are:

Apache HTTP server configuration

See the Torrus Web Interface Reference document for detailed instructions on Apache configuration.

Access Control Lists

By default, Torrus web interface requires user authentication. You can disable this by changing $Torrus::CGI::authorizeUsers to zero in your torrus-siteconfig.pl.

ACLs are controlled by acledit utility. See Torrus Manual pages for detailed description. Example:

torrus acledit --addgroup=staff --permit=DisplayTree --for='*'
torrus acledit --addgroup=staff --permit=GlobalSearch --for='*'
torrus acledit --adduser=jsmith --password=mysecretpassword \
  --cn="John Smith" --addtogroup=staff
torrus acledit --addgroup=admin \
  --permit=DisplayTree --permit=DisplayAdmInfo --for='*'

Cron Job

In order to clean old HTTP session data, it is recommended to run cleanup script in a cron job, once per day:

#min hour mday month wday    who     command
5    3    *    *     *       root    /usr/share/torrus/bin/cleanup

Startup script

The Torrus distriubution provides a System V init script which you can install in the init scripts directory (/etc/init.d on most Unixes or /usr/local/etc/rc.d/ on FreeBSD). The script torrus is created during the installation process in the subdirectory <init.d> of the distribution directory.

The init script reads some parameters from /usr/share/torrus/conf_defaults/initscript.conf, and /etc/torrus/conf/initscript.siteconf if the latter exists. By default, it makes the monitor daemons sleep for 20 minutes when they are launched simultaneously with collector daemons.

Also Torrus distribution comes with Systemd and Tmpfiles files, so if the OS has Systemd in place, System V init files should not be used. See the quick installation section (above) for more details.

Upgrading from Torrus version 2.x

Torrus 3.x is using a completely different back-end for its data structures: the old Torrus was using BerkeleyDB, while the new one uses a Git repository. Also run-time data is kept in a Redis database instance.

Torrus 3.0 is fully compatible with XML configuration and torrus-siteconfig.pl statements of Torrus 2.0.

The best approach in upgrading from version 2 is to delete the old database and cache directories (or simply everything in /var/torrus), and also the Torrus scripts and libraries (/opt/pm/torrus if you used /opt/pm as a prefix). Then, run the installation procedure and execute the XML compiler, and Torrus should be ready to run.

Upgrading within version 3.x

Follow these instructions when upgrading from previous Torrus version.

In the previous distribution directory, look up the config.log file. It contains the configure options that you used in previous installation.

Unpack the new release distribution.

Run ./configure with the same options you used before.

Stop the collector and monitor processes (usually by /etc/init.d/torrus stop or systemctl stop torrus).

Stop Apache process.

Run make install.

Start the collector and monitor processes.

Start Apache process.

Also it is recommended to re-compile your XML configuration.

Plugins

Plugin packages inherit a release numbering scheme as follows: a plugin release version is a 3-component number, where the first two numbers are the Torrus release version that is compatible with this plugin.

Planning the disk space

In a typical Torrus setup, there are two disk space consuming entities: the RRDtool data storage and the Git repository for configuration data.

RRDtool data files (or RRD's) are used to store the values that are gathered by the collector daemons. These are the most intensively updated files, so the disk speed is important here. If possible, it is better to spread the RRD files across several physical disks.

The Git repository is used to keep all the configuratiuon data for your datasource trees, and it also keeps some live status information. It's updated during XML compilation only. During normal Torrus working cycle, there are only read operations. The database is read quite intensively during collector initialization, but usually it's the CPU speed that causes more delay.

The Git repository needs double the space it occupies between the compilations. The compiler runs the garbage collector at the end of its work, and all unnecessary data is deleted.

Network performance tuning

In large installations, the SNMP collector experiences quite intensive input traffic bursts. Thus the UDP socket receive buffers should be adapted to sustain the load. By default, Torrus sets the UDP receiving buffer size, SO_RCVBUF, to 131071 bytes. This should fit most of installations. However, it's useful to check the network statistics if there are any UDP buffer overflows. On most systems, the commands netstat -s -p udp or netstat -s should show you these counters. The maximum buffer size is usually limited by a system kernel variable, and can be increased if needed. See $Torrus::Collector::SNMP::RxBuffer and its comments in torrus-config.pl for more details.

Author

Copyright (c) 2002-2017 Stanislav Sinyagin <ssinyagin@k-open.com>