From 37cf429995a86cbc87008e183b3c79b112d9b999 Mon Sep 17 00:00:00 2001 From: arjen Date: Mon, 27 Oct 2003 11:24:12 +0000 Subject: [PATCH] New user manual, based upon the draft TUTORIAL --- doc/manual.xml | 643 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 643 insertions(+) create mode 100644 doc/manual.xml diff --git a/doc/manual.xml b/doc/manual.xml new file mode 100644 index 0000000..0403023 --- /dev/null +++ b/doc/manual.xml @@ -0,0 +1,643 @@ + + + + + + + + + + Gnucomo - User's Manual + + Peter Roozemaal<mathfox@xs4all.nl> + Arjen Baart <arjen@andromeda.nl> + October 23, 2003 + + 0.1 + Andromeda Technology & Automation + De Winter Information Solutions + + + + + + + + +What is GnuCoMo ? + + +The aim of the GnuCoMo (GNU COmputer MOnitoring) project is to build a +set of applications that will help administrators to monitor networks as +a whole for errors, attacks and security-breaches in a very user-friendly +way. It is free (GPL) software, sources are available and you are free +to modify it to suit your needs. + + +GnuCoMo collects system and application logfiles from the monitored +systems (this can be computers, routers and other devices) and stores them +in a database. This collected data is further interpreted and analyzed +until reports and alerts are generated. Because GnuCoMo can combine data +from multiple sources and longer timespans than common IDSs it is able +to recognize attacks that previously remained undetected. + + +Remeber, this manual is work in progress. +Although it contains some usefull pointers to help you get started +with Gnucomo, it is still quite incomplete. +You can help us improve this manual and, in fact, Gnucomo itself, +by sending your comments and suggestions to the mailinglist: +gnucomo@dewinter.com. + + +
+About this Document + + +This document aims to guide you through the installation of GnuCoMo on +a Unix (Linux) workstation. When we have completed the installation this +document will give a quick guided tour through the functionality GnuCoMo +offers. The procedures described here aim at getting the GnuCoMo demo +up and running as quickly as possible and may not be suited for your +production environment. I assume you have some experience with installing +software on your system. + + + +This document is part of the Gnucomo package. Gnucomo is released under +the Gnu General Public License, see the file COPYING. + + +
+ +
+Reporting Bugs + + +If you think you have found a bug in GnuCoMo, this tutorial or the +other GnuCoMo documentation, please report it by sending an e-mail to +the mailinglist: gnucomo@dewinter.com. +Please include in your bug report: + + +the GnuCoMo version you've found the bug in +what you did to provoke the error +the output (or error message) you got +(if relevant) which output you expected +processor type and kernel version you use +the versions of the packages that gnucomo depends on + + + + +
+ + + +Installation + + +Most of the GnuCoMo installation can be done from an ordinary Unix +account; when you need root rights you will be told so explicitly. It is +however a good idea to create a gnucomo account. + + +
+Getting the software + + +GnuCoMo can be downloaded from the +GnuCoMo website. +If you read this file you are likely to have a +copy of GnuCoMo on your system. If you are serious about using GnuCoMo +it would be a good idea to periodicly check the website for updates +and bugfixes. + + +To be able to install and run GnuCoMo you'll need several other packages: + + + + PostgreSQL is the database we use for GnuCoMo. Most + linux distributions provide ready to install packages for + PostgreSQL. We need at least the postgresql, postgresql-server, + postgresql-libs and postgresql-devel packages. + If you want to compile PostgreSQL from source: go to the + PostgreSQL homepage + and download the sources via one of the ftp sites. We need libpq++ + support for GnuCoMo. + Though we appreciate the performance improvements that the + PostgreSQL 7.3 server provides, we recommend sticking to the + PostgreSQL 7.2 client versions for now; at least until we've + solved the problems with the libpq++ libraries and PostgreSQL 7.3. + + + + The C++ client interface for the PostgreSQL database server + is a separate project, not distributed with PostgreSQL. + This library replaces the old libpq++ library. + The place to find libpqxx is + + the Gborg website. + Make sure you install libpqxx in a default library path or + set your LB_LIBRARY_PATH environment variable to include + the directory in which libpqxx is installed. The default + for libpqxx is /usr/local/libpqxx/lib. + + + + PHP is used as programming language for gcm_deamon. If you + have packages you would want to install at least the php and + php-pgsql packages. + You can get PHP sources and documentation from the + PHP website. + If you're compiling yourself, don't forget + to include PostgreSQL support. + + + + We use XML for configuration that we can't (or don't want + to) store in the database and for documentation in XMLDOC + format. The libxml2 library usually comes with your linux + system and you would want to install both the libxml2 and + libxml2-devel packages. The libxml2 sources can be downloaded + via the XMLSoft site. + + + + It's not likely that you'll find precompiled AXE packages on + the net, so you'll have to compile from source. Get the AXE + sources from the + AXE page; + you'll need version 0.3 or better. + Instructions on compiling and installing AXE are given later in + this document. + + + + +The following packages are optional and provide additional functionality +to GnuCoMo: + + + + Recommended for encryption and signing of data that is transported + over the network. Not used at this moment. + + + + We use XMLDoc to process our documentation. Download + and installation instructions can be found on the + + XMLDoc site. + + + + [HELPME: basic instructions] + + + + A GUI based configuration tool named MalfisInter (mi) is being + worked upon. This tool is programmed in Python and requires XML + and TkInter support. + + + + + +
+ +
+Compiling + + +If you're lucky enough to find precompiled packages for your system and +have root permissions to install them, things are easy for you; otherwise +you would have to compile from source, which takes a bit more time if +you allready have the standard developer tools (C and C++ compilers, +make, (f)lex and yacc or bison) installed. You will need those tools +for GnuCoMo anyway. +You will also need the GNU automake and autoconf packages. + + +Another essential utility is the bzip compressor package; you will need +it to unpack the archives. bzip2 and bunzip2 come standard with Linux, +but may not be available on older Unix distributions. Sources for bzip2 +can be found at http://sources.redhat.com/bzip2/. +Most of the packages mentioned in this document come with detailed +compilation and installation instructions and I recommend to read the +README and INSTALL files before compiling and installing them. + + + +Compiling AXE + + +For compiling AXE you need the X-windows (X11) headers and +libraries. Under Linux you might need to install the XFree86-devel +package, but these headers are usualy available whenever the C compiler +is installed (also on propriatary unixes). +[TODO] + + + + + +Compiling GnuCoMo + + +The usual ./configure; make install will compile +the binary applications and install gcm_input and +logrunner into /usr/local/bin. +You will need to copy the web interface scripts and gcm_daemon +manually to the appropriate directories. + + + + +
+ +
+Setting up the database + +Gnucomo won't do anything usefull without a PostgreSQL database, so you will have to +set this up first. +Make sure you have the PostgreSQL server running and you have a valid userid +and password to access the server. +You also need to have the permission to create new databases. +If not, you will need to log in as the database administration user (usually +a user called postgres) and create a new user for the database. + + + +To create the initial Gnucomo database, you need to use the SQL script +create.sql, which is in the src/database +directory of the Gnucomo distribution. +Type the following commands to create the database: + + + + createdb gnucomo + psql gnucomo -f create.sql + + + +With the default installation of PostgreSQL on RedHat 8.0, +you will probably encounter an authentication problem when you try to +use the Gnucomo web interface. The problem will look somewhat like this: + + + + +Warning: pg_connect() unable to connect to PostgreSQL server: +FATAL 1: IDENT authentication failed for user "arjen" + + + + +Refer to PostgreSQL Administrator's guide, Chapter 4: Client Authentication. +You probably have this line in the /var/lib/pgsql/data/pg_hba.conf: + + + local all ident sameuser + + +(I know RedHat 8.0 does this). You need to change this into: + + + local all password + + +This tells PostgreSQL to allow any UNIX user to log into the database +as any database user on a local socket, using his database password. + + +
+ +
+The Gnucomo configuration file + + +All Gnucomo applications use a single configuration file, named +gnucomo.conf by default. +There are three places where this configuration file is stored. +Gnucomo applications first look for the configuration file in /etc, +and if the file is not found there, it should be in +/usr/local/etc. +Furthermore, you can have an additional configuration file in your home +directory, named $HOME/.gnucomo.conf. +The configuration file in /etc or in /usr/local/etc +holds the system-wide parameters. +The configuration file in a user's home directory may hold user-specific +preferences. +The system-wide configuration is mandatory; any user-specific configuration +is optional. + + +The configuration parameters are organized as a two-level tree, written +in XML. +The first level of the tree constitutes the 'sections', referring to a +set of related paramaters. +The second level contains the parameters themselves. +Each parameter is a single value, i.e. there are no provisions to create +structured parameters. +Here is an example of what the Gnucomo configuration file may look like: + + + <?xml version='1.0'?> + <gnucomo version='0.0.8'> + <database> + <type>PostgreSQL</type> + <name>gnucomo</name> + <user>arjen</user> + <password>guess-again:-)</password> + </database> + <logging> + <method>file</method> + <destination>/var/log/gcm_input</destination> + <level>0</level> + </logging> + </gnucomo> + + + +At the root of the tree is an XML element, called gnucomo. +The direct children of the root element in the example above are +database and logging. +These are the 'sections', corresponding to different aspects of the Gnucomo +system. +As you may have guessed, the database section holds information +we need to access the Gnucomo database on the PostgreSQL server and the +logging section specifies how much logging we want the +Gnucomo applications to generate and where we want that information to go. + +
+ +
+Getting the web interface up and running + +Your primary view onto Gnucomo is through the web interface. +Gnucomo uses a web server, e.g. Apache, with PHP scripts to implement +its user inetrface +At the present, the web interface is the only way to manage the +information in the Gnucomo database and view the results that +Gnucomo produces. + + +To be able to use the web interface, you need to copy the PHP +scripts from the src/web and src/phpclasses +directories into a directory accessible through the web server. +On most Red Hat Linux systems, this is usually in +/var/www/html. + + +
+ +
+And now... + +If you like GnuCoMo [TODO] +Report success under Non-Linux [TODO] + +
+ + + +Using Gnucomo + + +Once you have the database and the web interface set up, Gnucomo is almost +ready to go. +To put Gnucomo to work, you need to enter at least one object into the database +and create your methods to feed information to gnucomo. + + +
+The initial entries + + +Gnucomo maintains information about monitored systems such as servers, +work stations, switches and that sort of thing. +In Gnucomo, these are called objects. +The objects are distinguished by their name, usually +a hostname in your network. Or, better still, a fully qualified name +on the internet. +You can use the web interface to create an object by logging +in and selecting the picture that looks like a computer cabinet +in the button bar at the top. +Enter the object's name, for example server.gnucomo.org, +and click the Create button. + + +
+ +
+Feeding log files + + +Once an object is known by Gnucomo, i.e. the object's name is stored in the +database, you can start to fill the database with log entries. +Log entries are stored by a variety of programs into one or more log +files on the object's harddisk. +For example, system messages on most Linux distributions are put +in a file called /var/log/messages. +There are several ways to transfer the contents of a log file into the +Gnucomo database, each of which involves gcm_input. +Gcm_input is the Gnucomo application that reads various kinds +of raw input and tries to scan for as much information as possible. +The usefull information that is extracted from the input is stored +into the Gnucomo database. + + + +The most direct way to invoke gcm_input maually and read +the log file through its standard input: + + gcm_input -h server.gnucomo.org </var/log/messages + +You'll have to explicitly specify the hostname of the object because +there is no way for gcm_input to know where the log +file comes from. +Take care not to feed the same log file another time through +gcm_input. +This will lead to duplicate log entries in the Gnucomo database. + + +Feeding information into the Gnucomo database can also be done through +a mail server, such as sendmail. +Create a mail alias that invokes gcm_input as a mailer program. +For example: + + gnucomo: "|gcm_input" + +With such an email alias, Gnucomo clients can simply send log files and +other reports to this email address: + + mail gnucomo@server.gnucomo.org </var/log/messages + + + +The third method to feed log files into the Gnucomo database is by +using logrunner. + +
+ +
+Viewing results + +Use the web interface to review log entries, notifications and parameters. + +
+ + + + +Managing system parameters + + +Besides log entries which represent the transient events that occur in +an object, Gnucomo also maintains the state of the object. +The state of an object is defined by the set of parameters and +their values. +Examples of parameters are the file systems, users, processes, installed +packages, and so on. +By managing those parameters with Gnucomo, you can hold a firm grip on +the state of your systems. +Especially on large sites with many workstations, servers, switches +and other objects, managing the state of all those systems is not +a trivial task. +The following sections explain how Gnucomo can help. + + +
+Installed packages + + +Gnucomo can maintain a list of all packages installed on an object. +This list is stored in the database as parameters of class +package. +Each package parameter has one property: the version number +of the package. +Gnucomo keeps an historical record of the version number for each package, +so you will know when the version of a package changes. +Furthermore, you can compare the installed packages of two objects, so +you can easily find the differences between two installations. + + + +To maintain the installed packages of an object, all you need is to +feed a complete list into the Gnucomo database through gcm_input. +The package list is a text file with a package on each line of the form +packagename-version. +Not the dash ('-') between the name and the version. +On many Linux systems that use the RedHat Package Manager (RPM), such +a list is easily obtained and fed into gcm_input: + + + rpm -qa | gcm_input -h example.gnucomo.org + + +You will have to repeat this procedure at regular intervals. +Each time you feed a new package list to Gnucomo, Gnucomo will +compare the new list with the package parameters in the database +and notify you of any changes. + + +For keeping your systems up-to-date, it is often convenient to maintain +a repository of the most recent packages. +Gnucomo can help you with this task. +To enter the list of packages and updates from a distribution into the +Gnucomo database, you can create a virtual object and +create the package parameters for this object. +For example, to maintain a list of RPMs for RedHat Linux 7.3, create an +object (using the web interface) with the name redhat-7.3. +Using this object, enter the first batch of package parameters from a directory +listing of the CD-ROM. +However, you can not use the output from ls directly as +input for gcm_input. +You need to strip off two siffixes off the filenames to make it look like +a rpm -qa output. +The following script will do just that: + + + +#!/bin/sh +# +# Turn an 'ls' listing of RPM files into an 'rpm -qa' listing +# Reads a list of filenames, possibly preceeded by a directory and +# strips the directory path from the beginning and the two suffices +# from the end of each filename. For example, the name +# "/mnt/cdrom/RedHat/RPMS/kernel-2.4.20-13.7.i686.rpm" gets turned +# into a simple "kernel-2.4.20-13.7". + +while read filename +do + case $filename in + *.src.rpm) + ;; + + *) + filename=`basename $filename .rpm` + case $filename in + *.athlon) + rpm=`basename $filename .athlon` + ;; + *.i386) + rpm=`basename $filename .i386` + ;; + *.i486) + rpm=`basename $filename .i486` + ;; + *.i586) + rpm=`basename $filename .i586` + ;; + *.i686) + rpm=`basename $filename .i686` + ;; + *.noarch) + rpm=`basename $filename .noarch` + ;; + esac + echo $rpm + ;; + esac +done + + + +Suppose this script is stored as ls-rpm, you can apply it +like this: + + + ls /mnt/cdrom/RedHat/RPMS | ls-rpm | sort | uniq | gcm_input -h redhat-7.3 + + +You can repeat this command to enter additional packages into the database. +For example, to add packages from other CD-ROMs or from a directory where +you keep downloaded updates. +Remember, however, to use the -i option for "incremental" +with gcm_input. +Without this option, gcm_input will regard the list of packages as a full +list and remove any parameters that are not present in the new list. + + +With a list of available packages for a specific distribution in the Gnucomo +database, along with the list of packages installed on your systems, you +can easily obtain an overview of which packages you need to update. +To make this overview, use the web interface to compare the packages +on an object with the packages on the virtual object. + +
+ + + + + -- 2.11.0