Logrunner is statically linked for easy deployment.

[gnucomo.git] / ChangeLog

gcm_input
----------

 - logrunner is statically linked so it can be copied to different systems,
   32 bit or 64 bit, without library dependencies.

gcm_maintenance
-----------

 - Cleanup abuse records only for subnets smaller than /16.

gcm_daemon
-----------

 - Resolve a hostname before adding to the abuses.

Database
---------

 - Added a new group daemon and created additional permissions to the database.
 - Cleaned up the COLUMN recognized from the TABLE logng

web interface
-------------

 - Put the password for logging into the database between quotes (').
 - When removing an object, clean up logs, parameters and notifications.



Dec 12, 2007 - Release 0.0.12
====================================

gcm_input
---------

- New options added for logrunner:
   -l <file>: Read input from logfile <file>.
   -s <file>: Use <file> as the status file instead of the default.
   -v Verbose output

- Logrunner outputs debug information only when the -v option
  is added on the command line.

- Buffer overflow problem in logrunner with input lines of more than 4096
  characters fixed.

gcm_daemon
-----------

- Added a new script gcm_maintenance.php to cleanup the database
  and check referential integrity. Purging old log entries is
  removed from the gcm_daemon script.

web interface
-------------

- Added fields to edit the range of a DYNAMIC property.

- The index.html page in now obsolete and redirects to index.php

- Added an option to perform one action on multiple notifications at once.

- When blocking a subnet, maintain the references to the log table from
  all IP addresses in that subnet.

Nov 22, 2007 - Release 0.0.11
====================================

Database
---------

- Added the class definition for a filesystem parameter.
- New issue type: 'property out of range'.

gcm_input
---------

- Logrunner can select a specific host from a logfile, in case
  syslogd acts as a log server for multiple systems.
  This adds a new parameter (logile/fromhost) to the configuration
  file.

- Logrunner uses the official hostname (FQDN) instead of the
  hostname returned by gethostname().

- Handling of parameters is greatly improved.
  When creating a new parameter from an XML report which is fed into
  gcm_input, the class definition is used as a template to fill in
  the default values for the properties.
  The class is also used as a template when a new property is added
  to an existing parameter.

- DYNAMIC properties are now handled properly. Instead of making a
  'changed property' notification, the value is checked against the
  defined range for the property. An 'out of range' notification
  is created when this condition is detected.

- Added a new filter which can directly read the output
  of the UNIX df command. A brief description is added in
  the user manual.

- Bugfix: Segmentation fault when reading an rpm package list
  with empty lines.

- New program: spamdetect.
  Expirimental utility to log manually reported spam and have
  Gnucomo detect the spammer's IP address.

- Added a number of usefull scripts, mainly for use in Gnucomo clients.

web interface
-------------

- The buttonbar at the top of each page is now a fixed 'div' element
  instead of a framed page.
  Contributed by Edwin Nadorp.

PHP classes
-----------

- Added a PHP5 module for the configuration class.

Test
----

- The test script, gcmtest can run individual tests as opposed to running
  all tests in sequence.

Oct 19, 2007 - Release 0.0.10
====================================

- Improved method for finding PostgreSQL libraries in configure script.

Database
---------

- Fixed field declaration for PostgreSQL 7.4
- New tables: log_abuse, object_abuse and service_pattern
- Added general service patterns in the service 'ANY'
- Added new issue types.
- Log_adv and derived tables removed.

logrunner
---------

- Status files in /var/lib
- Rewrote most functions and datastructures into C++
- Use the Gnucomo (XML) configuration file

gcm_input
---------

- Accept '/' and '.' characters within the service name in a system log.
  When extracting the service name, the '/' character is not considered
  part of that name, though.
- Textual changes in parameter notifications
- Experimental start of database OO abstraction layer.
- The '-' character is part of the service name.
- Skip the Email header when reading XML input.

gcm_daemon
-----------

- Bugfixes in the analysis of sendmail logs.
- Send email about open notifications to an object's users.
- Added log analysis for spam and abuse in sendmail log entries.
- Added pattern check on log entries with the service_pattern table.
- Create notifications from log entries with pattern matching.
- Record the date when adding the number of abuses for an IP address
  in the abuse list.


web interface
--------------

- Added a form to edit user data.
- Added a view for the log analysis with a link from the log page.
- New page: Parameter classes administration.
- New page: Abuse list
- Added an interface to edit check patterns to the services page.
- Manually edit parameters.
- View logs from abusing IP addresses.
- Added an interface for editing issues.
- In the abuse list, IP addresses can be whitelisted.
- Improved interface for editing check patterns.

Test
----

- Added test data for Problem Report 15, 16 and 17

Documentation
---------------

- Added a list of related projects and introduced the concept of
  dynamic parameters in the manifest.

Dec 24, 2003 - Release 0.0.9
====================================

Database
---------

- Changed the type of log_adv_daemon_email.delay and log_adv_daemon_email.xdelay
  from time to interval. These delays can be more than 24 hours.

libgnucomo
----------

- Catch an exception if we can not setup a database transaction.


gcm_input
---------

- SQL_Escape(): Backslashes are correctly escaped with another backslash
- XML_Entities(): transform non-ASCII characters into hexadecimal entities.
- Do not add another parameter_notification record is the notification
  already exists for that parameter.
- Catch exceptions from the database library.
- Major redesign. All input is handled through XML. Raw input data is first
  transformed into an XML document for further processing.
  A collection of polymorphic classes handle the transformation of various
  input formats into XML.
- Classifying input data is done with a finite improbability calculation.

gcm_daemon
-----------

- Create separate notifications for different objects in service_check().
  Fixes problem report 23.
- Change the notation for delays from "days+hh:mm:ss" to "days hh:mm:ss. This
  fixes problem report 14.
- In linux_daemon_sendmail(), check the existence of '=' before using it as a
  field separator. Fixes problem report 15.


web interface
--------------

- Optionally show or hide removed parameters from the parameter
  comparison page.

Documentation
---------------

- Added a User Manual, based upon the draft TUTORIAL


Sep 04, 2003 - Release 0.0.8
====================================

Database
---------

- Fixed a few typos in the database creation script.
- BUGFIX: Secondary indices on log_notification were unique.
- Additional information in the 'usr' table: 'display_name' and 'email'.
- Added new issues and services.

gcm_input
---------

- Fixed a namespace problem in message.cpp
- Fixed a gcc 2 vs. gcc 3 problem in gcm_input.cpp
- Reject log entries that are found to be invalid.
- A date without the time for the '-d <date> option will
  assume midnight on that date.

gcm_dameon
----------

- In PHP, the method configuration::read() will also read
  a user-specific configuration from the home directory if a 
  PHP script is not run through the web server.
- Accept command argument '-c config' to use an alternate
  gnucomo configuration.
- New PHP methods:
   db::Result() - Returns the result from the last query.
   db::Field()  - Returns the value of a single field.
   db::new_notification() - Create a new notification in the database.
- Check the log table against the servies running on an object and
  create notifications if a service is not supposed to be available
  or is not known at all.

libgnucomo
----------

- Changed the gnucomo_database class to the new PostgreSQL
  library, libpqxx
- Overloaded gnucomo_database::Field() to include the Result
  from a query as an argument.

web interface
--------------

- New page to enter and modify services.
- The objects page provides an interface to edit the list of
  services and users for an object.

Test
----

- Reduced the amount of output from test scripts.
- Added output data that is expected from test scripts.
- New test: tests the database upgrade with gcm_daemon
- Overall test script: gcm_test. Runs all tests in sequence.


Aug 15, 2003 - Release 0.0.7
====================================

Database
---------

- Added index to the history table to improve performance.

gcm_input
----------

- Added a new section 'logging' with three configuration parameters:
   method       - Output method to use for logging.
   destination  - Name of the log output destination.
   level        - Log level: Verbose output if greater than 0.
- Added '-i' option for incremental parameter updates.
- Debug output to the log stream instead of cerr.
- Fixed namespace problems in XPath searches of the DOM.
- Moved string utility functions to a separate file.
- Different kinds of log files are parsed by a collection of objects
  of different classes, derived from the base class line_cooker
  Depending on the message content or the message_type element in
  XML, one of these objects is selected.
- Logrunner is integrated with gcm_input. Although its functionality
  is still limited, a connection between logrunner and gcm_input
  is beginning to form.

gcm_daemon
----------

- BUGFIX: Print an error message if a parameter does not have
  any history.
- BUGFIX: undefined variables and indices when processing sendmail logs.

web interface
--------------

- BUGFIX: Convert special characters for HTML (<, >, and &) into
  their entities.
- BUGFIX: Removed parameters were somtimes shown on the wrong side
  of the parameter difference page.
- Added performance measurement to the parameters page.
- Added editing of detailed object information.

Jul 15, 2003 - Release 0.0.6
====================================

- Gcm_input reads IRIX system logs.
- Gcm_input exits without reading any input if the database connection fails.
- Gcm_input extracts the hostname out of the 'From:' or 'Message-Id:' line
  of an email header.
- New member C++ function gnucomo_database::is_conected().
- New columns in the table 'log_adv_daemon_email': size, pri, relay,
  status_details and dsn.
- Added several indices for the table 'log_adv_daemon_email'.
- In phpclasses/db.class.php: Added the database connection string as
  an argument to the function copy_db_class.
- Gcm_daemon processes logs from sendmail.
- Fixed the PHP member function db::db_connect(). The Postgres connection
  string is now passed as an argument to that function.
- Gcm_input reads cooked log entries from an XML input stream.
- New database tables: notification_check, notification_check_buffer,
  notification_check_line and object_statistics.
- PHP function db::query() returns the result index.
- PHP function db::num_rows() accepts a result index as argument
  (default = 0 -> use result from the previous query).
- New PHP function db::fetch_object().
- Gcm_daemon gathers statistics on parameters, notifications, etc. for all objects.
- The Objects web page uses the new object_statistics table.
- The Log web page shows the log one day at a time.
- Removed parameters are displayed in a shaded style.

Feb 21, 2003 - Release 0.0.5
====================================

- The notifications web page keeps a record of each time a notification
  is displayed. Either on a listing or in detail.
- Improved the table layout of the objects web page.
- In the users page of the web interface, you can add a new user and
  make him/her a member of a group.
- Users can change their password through the users page of the web interface.
- Gcm_input also detects packages that are removed from the system.
- In gcm_input, determining the version number of a package in a RPM
  list is improved. Only the last one or two parts of the string that
  begin with a '-' and a number are considered the version.
- The C++ function gnucomo_database::new_notification() uses the SQL
  function currval() to obtain the identification number
  of the most recently created notification.
- The C++ class gnucomo_database prints the query on cerr, along
  with the error message when the query results in an error.
- Removed the 'Change Password' image from the main menu.
  Changing passwords is integrated in te users page.
- Added the 'form' class on table and td elements. This class is intended
  for borderless tables that are used to layout HTML forms.
- A new style 'h2.error' for error reports in the stylesheet.
  Put borders around the tables and cells in the web interface.
  Added right-alignment for numbers and fixed width for timestamps.
- Gcm_daemon maintains the last notification identifier.
- Setup the groups and database permissions. The three groups
  are: view, ops and admin.
- Added two new entries to the action table.
- All web interface pages use the page class. This provides for a
  uniform session and database handling.
- Use our own error handler for PHP errors and warnings in the web interface.
- Added log, notification and parameter counters to the 'object' table.
  Counting these things at the time a user interface needs them is
  too slow. Other programs, like gcm_daemon en gcm_input should prepare
  these counters for quick retrieval.
- Show the total number of Log entries, parameters and notifications
  on the object page of the web interface.
- Added new tables to the database : log_adv_daemon and log_adv_daemon_email.
- Made gcm_daemon a 'proper' executable.
- Paths of included files in PHP scripts are more flexible.
- Gcm_daemon recognizes important start and stop events of various daemons
- Gcm_daemon checks for exiting daemons in the log entries.
- Directory structure of PHP scripts reorganized. PHP scripts that are included in
  both gcm_daemon and the web interface are now in the directory src/phpclasses.


Feb 05, 2003 - Release 0.0.4
====================================

- Added style parameters for diff-like table views and textarea elements.
- Display and handle notifications in the web interface.
- Display the difference of all package class parameters for two objects
  in the web interface.
- New PHP class added: 'page'. This is a (sort-of abstract) base class
  for all Gnucomo web interface pages. It hanldes the session, login,
  opening the database and the head and tail of the web page.
- gcm_input creates notifications when a new package is discovered
  in a 'rpm -qa' list or when the version of a package is changed.
- Bug fix: action.statuscode was in upper case.
- Added new tables to the database: parameter_class and parameter_notification.
- New C++ function: gnucomo_database::new_notification(). retruns the id number
  of the newly created notification record.
- Added new records to the 'type_of_issue' table.
- Changed semantics of actionid 9 in the 'action' table.
- Bugfix in gcm_input: The hostname in a system log may contain
  digits as well as letters.
- Added new fields to the 'type_of_issue' table.

Dec 06, 2002 - Release 0.0.3
====================================

- Directory structure is reorganized and prepared for GNU configure.
- gcm_input sets the value of log.processed to FALSE when inserting a
   new log entry into the database
- When a syslog entry arrives from last year, gcm_input subtracts one from the
   year of arrival to create the year of the log entry.
- Gcm_input reads output from "rpm -qa" and enters packages in the parameter table.
- Changes to log and log_adv tables described in the manifest.
- Added description of gcm_daemon in the design.
- Added new program (in PHP): gcm_daemon.
- Added a check on the database version in gcm_damon.
- Database-login in gcm_damon can be done using TCP/IP or UNIX-sockets.
- Gcm_daemon processes iptables-records including ICMP support.
- Returning values with ICMP (bounced) are seperate and linked to the same logfile.
- Which each new version records that haven't been recognized are processed again
  through gcm_daemon.
- Added program-specific database user and password in config file.
- Added new arguments to gnucomo_config::Database(): user and password.
  If empty, default values are taken from the config file.
- Implemented a slightly functional web interface that shows some
  output from the database.