<author>Brenno J.S.A.A.F. de Winter, De Winter Information Solutions</author>
<author>Arjen Baart, Andromeda Technology & Automation</author>
- <date>August 28, 2002.</date>
+ <date>December 12th, 2002.</date>
<docinfo>
- <infoitem label="Version">0.19</infoitem>
+ <infoitem label="Version">0.24</infoitem>
</docinfo>
</titlepage>
<chapter>
<heading>About this document.</heading>
-<para>This document describes the technical specifications for the Gnucomo project. It will describe the functionality achieved, design specifications and choices made. The document will be the manifest for the developers to work in the same direction and not run into unneeded disappointments.</para>
+<para>
+<LaTeX command='\setlength{\parindent}{0cm}'/>
+<LaTeX command='\setlength{\parskip}{0.4cm}'/>
+This document describes the technical specifications for the Gnucomo project.
+It will describe the functionality achieved, design specifications and choices made.
+The document will be the manifest for the developers to work in the same direction
+and not run into unneeded disappointments.
+</para>
<section>
<heading>History of the document.</heading>
-<table cpos='lp{3cm}lp{4cm}'>
+<table cpos='lp{3cm}lp{5cm}'>
<thead>
- <col> Version </col> <col> Author </col> <col> Date </col> <col> Remarks </col>
+ <col>Version</col><col>Author</col><col>Date</col><col>Remarks</col>
</thead>
<row>
- <col> 0.l </col> <col> Brenno de Winter </col> <col> July 11th 2002 </col>
+ <col>0.l</col><col>Brenno de Winter</col><col>Jul 11, 2002</col>
<col>
Initial version
</col>
</row>
<row>
- <col> 0.11 </col> <col> Arjen Baart </col> <col> July 12th 2002 </col>
+ <col>0.11</col><col>Arjen Baart</col><col>Jul 12, 2002</col>
<col>
Additional guidelines and dataflow diagram.
</col>
</row>
<row>
- <col> 0.12 </col> <col> Brenno de Winter </col> <col> July 15th 2002 </col>
+ <col>0.12</col><col>Brenno de Winter</col><col>Jul 15, 2002</col>
<col>
Database design added
</col>
</row>
<row>
- <col> 0.13 </col> <col> Arjen Baart </col> <col> July 16th 2002 </col>
+ <col>0.13</col><col>Arjen Baart</col><col>Jul 16, 2002</col>
<col>
Entity-relationship model added.
</col>
</row>
<row>
- <col> 0.14 </col> <col> Brenno de Winter </col> <col> July 17th 2002 </col>
+ <col>0.14</col><col>Brenno de Winter</col><col>Jul 17, 2002</col>
<col>
Based on feedback. Small changes to the datamodel and finishing touches
in lay-out of the tables. Added some examples.
</col>
</row>
<row>
- <col> 0.15 </col> <col> Brenno de Winter, Arjen Baart </col> <col> July 21st 2002 </col>
+ <col>0.15</col><col>Brenno de Winter, Arjen Baart</col><col>Jul 21, 2002</col>
<col>
Additional feedback processed, indexes added, ERD added and SQL-script created.
</col>
</row>
<row>
- <col> 0.16 </col> <col> Brenno de Winter </col> <col> August 7th 2002 </col>
+ <col>0.16</col><col>Brenno de Winter</col><col>Aug 7, 2002</col>
<col>
Communication handling added.
</col>
</row>
<row>
- <col> 0.17 </col> <col> Brenno de Winter </col> <col> August 11th 2002 </col>
+ <col>0.17</col><col>Brenno de Winter</col><col>Aug 11, 2002</col>
<col>
<para>* Description of the client-side for communications.</para>
<para>* Several updates to the database descriptions, drawings added.</para>
</col>
</row>
<row>
- <col> 0.18 </col> <col> Brenno de Winter, Peter Roozemaal </col> <col> August 15th 2002 </col>
+ <col>0.18</col><col>Brenno de Winter, Peter Roozemaal</col><col>Aug 15, 2002</col>
<col>
<para>* Review done by Peter Roozemaal: adjusted intro and several clarifications made</para>
<para>* Arjen Baart: Adjustments to database drawings</para>
</col>
</row>
<row>
- <col> 0.19 </col> <col> Arjen Baart </col> <col> August 27 2002 </col>
+ <col>0.19</col><col>Arjen Baart</col><col>Aug 27, 2002</col>
<col>
Conversion to XMLDoc
</col>
</row>
+ <row>
+ <col>0.20</col><col>Arjen Baart</col><col>Oct 20, 2002</col>
+ <col>
+ Minor layout improvements
+ </col>
+ </row>
+ <row>
+ <col>0.21</col><col>Arjen Baart</col><col>Nov 07, 2002</col>
+ <col>
+ Installation instructions added.
+ Combined chapters 4 through 7 into one chapter (4).
+ </col>
+ </row>
+ <row>
+ <col>0.22</col><col>Arjen Baart</col><col>Nov 15, 2002</col>
+ <col>
+ Added parameters of monitored objects.
+ </col>
+ </row>
+ <row>
+ <col>0.23</col><col>Brenno de Winter</col><col>Dec 6, 2002</col>
+ <col>
+ Added new elements to the database
+ </col>
+ </row>
+ <row>
+ <col>0.24</col><col>Brenno de Winter</col><col>Dec 12, 2002</col>
+ <col>
+ Updates to the database to reflect the most recent changes.
+ </col>
+ </row>
</table>
</section>
This evolves to the following list of functions gnucomo can provide:
</para>
<itemize>
+ <item> Intrusion detection </item>
+ <item> Detecting hacker attempts </item>
+ <item> Early detection of system failures </item>
+ <item> Exhaustion of system resources </item>
+ <item> Capacity planning for future expansion </item>
+ <item> Spotting bottlenecks in a system. </item>
+ <item> Verifying system integrity </item>
+ <item> Assistance with troubleshooting </item>
+ <item> Perform post-mortem forensics </item>
+ <item> Incident response system. </item>
+</itemize>
+</chapter>
+
+<chapter>
+ <heading>Decisions for the overall system.</heading>
+
+<para>
+In order to get the project running we have to make some decisions before
+we can start. Of course are the decisions always open for review,
+but initially our main aim is to get a system running. This doesn't
+mean that we allow a lesser architecture, but more that we create an
+environment that will lead to results.
+</para>
+<para>
+The following decisions apply to the system in general:
+</para>
+<itemize>
<item>
- Intrusion detection
+ The major part of the system will be used for security features.
+ The solution itself has to be secure.
</item>
<item>
- Detecting hacker attempts
+ In general we strive to be as platform and application independent as possible.
+ However to achieve software to get ready and to assure progress some selections
+ will have to be made excluding options for applications<footnote>An example is
+ in the database. By using stored procedures and triggers MySQL cannot be used.
+ However the data integrity at that point is more important than the ability to
+ use MySQL.</footnote>. Where possible we will enable ports to other applications
+ and platforms.
</item>
<item>
- Early detection of system failures
+ The computers collecting the data will only be Linux machines,
+ with future support for other Unices. However it is not aim our
+ to port this part of the project to other platforms just now.
</item>
<item>
- Exhaustion of system resources
+ The database system will make heavy use of stored procedures and
+ triggers and thus lock out some less feature-full database.
</item>
<item>
- Capacity planning for future expansion
+ Although we're not married to a database system initially this will be
+ PostgreSQL only so that we can build for results. The code however
+ shouldn't be build to deliberately lock outer systems.
+ Other database that might be interesting to add: DB2, Oracle
+ (if the rainfall of security advisories seems to be over) and Informix.
</item>
<item>
- Spotting bottlenecks in a system.
+ The interface for the user will be a web-interface written in PHP
+ with PostgreSQL-database access. Despite the fact that initially
+ this will be the preferred way to communicate with our system,
+ other interfaces are welcome and should be supported.
</item>
<item>
- Verifying system integrity
+ The technologies used for the daemons on the central site and remote
+ site are open for discussion. A decision is made when it is clear what
+ exactly will be needed to code.
</item>
<item>
- Assistance with troubleshooting
+ The original logfiles on systems will not be harmed in any way,
+ but will be saved the way they presented to gnucomo.
</item>
<item>
- Perform post-mortem forensics
+ In the database processed data as well as raw data will be stored.
</item>
<item>
- Incident response system.
+ Our main aim is to ease the life of the administrator when dealing with
+ the symptoms of a machine so that he doesn't miss important notifications.
+ The project will be made around detection. We won't focus on making a
+ maintenance applications but solely on monitoring, because other applications
+ like webmin or linuxconf do already deliver such functionality.
+ We are complementary to those applications. This doesn't mean that
+ the system cannot send SMS'es, e-mails or make alarms sound.
+ </item>
+ <item>
+ Any output of the logbook will be stored and sent within a certain
+ interval which can be set. By doing that the mechanism will not flood
+ the total mechanism and overload machines.
+ </item>
+ <item>
+ When applications for gnucomo we will try and do the non-server part
+ as platform independent as possible. This in order to prevent writing
+ a same application multiple times. For instance one for Unices, one
+ for Macintosh and one for Microsoft.
+ </item>
+ <item>
+ To ensure the quality of software and to prevent any unwanted functionality
+ rolling into the project all code will be reviewed by one of the lead members
+ before it is accepted. Since this project is aimed at throughput speed no
+ formal procedures will be enforced, but we promise not to let software in
+ that hasn't been checked.
</item>
</itemize>
-</chapter>
-<chapter>
- <heading>Decisions for the overall system.</heading>
-<para>In order to get the project running we have to make some decisions before we can start. Of course are the decisions always open for review, but initially our main aim is to get a system running. This doesn't mean that we allow a lesser architecture, but more that we create an environment that will lead to results. </para>
- <para>The following decisions apply to the system in general:</para>
- <itemize>
- <item>
- The major part of the system will be used for security features.
- The solution itself has to be secure.
- </item>
- <item>
- <para>In general we strive to be as platform and application independent as possible. However to achieve software to get ready and to assure progress some selections will have to be made excluding options for applications<footnote>An example is in the database. By using stored procedures and triggers MySQL cannot be used. However the data integrity at that point is more important than the ability to use MySQL.</footnote>. Where possible we will enable ports to other applications and platforms.</para>
- </item>
- <item>
- The computers collecting the data will only be Linux machines, with future support for other Unices. However it is not aim our to port this part of the project to other platforms just now.
- </item>
- <item>
- The database system will make heavy use of stored procedures and triggers and thus lock out some less feature-full database.
- </item>
- <item>
- Although we're not married to a database system initially this will be PostgreSQL only so that we can build for results. The code however shouldn't be build to deliberately lock outer systems. Other database that might be interesting to add: DB2, Oracle (if the rainfall of security advisories seems to be over) and Informix.
- </item>
- <item>
- The interface for the user will be a web-interface written in PHP with PostgreSQL-database access. Despite the fact that initially this will be the preferred way to communicate with our system, other interfaces are welcome and should be supported.
- </item>
- <item>
- The technologies used for the daemons on the central site and remote site are open for discussion. A decision is made when it is clear what exactly will be needed to code.
- </item>
- <item>
- The original logfiles on systems will not be harmed in any way, but will be saved the way they presented to gnucomo.
- </item>
- <item>
- In the database processed data as well as raw data will be stored.
- </item>
- <item>
- Our main aim is to ease the life of the administrator when dealing with the symptoms of a machine so that he doesn't miss important notifications. The project will be made around detection. We won't focus on making a maintenance applications but solely on monitoring, because other applications like webmin or linuxconf do already deliver such functionality. We are complementary to those applications. This doesn't mean that the system cannot send SMS'es, e-mails or make alarms sound.
- </item>
- <item>
- Any output of the logbook will be stored and sent within a certain interval which can be set. By doing that the mechanism will not flood the total mechanism and overload machines.
- </item>
- <item>
- When applications for gnucomo we will try and do the non-server part as platform independent as possible. This in order to prevent writing a same application multiple times. For instance one for Unices, one for Macintosh and one for Microsoft.
- </item>
- <item>
- To ensure the quality of software and to prevent any unwanted functionality rolling into the project all code will be reviewed by one of the lead members before it is accepted. Since this project is aimed at throughput speed no formal procedures will be enforced, but we promise not to let software in that hasn't been checked.
- </item>
- </itemize>
- <para/>
</chapter>
<chapter>
- <heading>Overall system.</heading>
+ <heading>Overall system Architecture</heading>
-<para>The overall systems aims to make maintenance data better accessible and by doing that lowering the barrier to be on alert for intrusions, system failure and other misery that can happen to computer systems. Also more systems can be monitored and even focus can be placed on desktop computers, something that nowadays rarely happens. Since the solution is only aimed at monitoring (with some responses possible) other people can watch their system(s) without being Administrator<footnote>This can be interesting for people using this service and not doing their own maintenance (smaller companies). If technical action needs to be taken a warning can be send to an administrator. This saves costs and time.</footnote>.</para>
- <para>The main system will know to sides:</para>
-
- <itemize>
- <item> Central Application - server </item>
- <item> Monitored System - client </item>
- </itemize>
-
- <para>The project has been setup as a two sided system in order to be able to guard many computers at the same time. However it may be obvious that both sides of the application can very well be installed on a single system.</para>
- <para>When events<footnote>A signal can be the outcome of process that finished, logbook entries, warnings from intrusion detection systems, etc. </footnote> happen they will be stored in a file. When this file is delivered to a certain directory a daemon will detect this and start the transfer of the file. The file will be transferred to the central application or the client. This transfer will be triggered immediately after a process has finished or with a certain time-interval when it concerns a logbook. All output will be placed in a directory where the daemon detects it and ensures the transport. For transport currently only two mechanisms will be supported:</para>
- <itemize>
- <item>
- Encrypted file copy relying on the SSH protocol. On Unix-based systems this will scp (secure copy).
- </item>
- <item>
- E-Mail. The e-mail is encrypted and signed using gpg and then sent to the server. Since the file format will identify the type-of-output the Subject-field of the e-mail is not really needed.
- </item>
- </itemize>
+<para>
+The overall systems aims to make maintenance data better accessible and by
+doing that lowering the barrier to be on alert for intrusions,
+system failure and other misery that can happen to computer systems.
+Also more systems can be monitored and even focus can be placed on desktop computers,
+something that nowadays rarely happens. Since the solution is only aimed at
+monitoring (with some responses possible) other people can watch their system(s)
+without being Administrator<footnote>This can be interesting for people using
+this service and not doing their own maintenance (smaller companies).
+If technical action needs to be taken a warning can be send to an administrator.
+This saves costs and time.</footnote>.
+</para>
+<para>
+The main system will know to sides:
+</para>
- <section>
- <heading>Priority mechanism.</heading>
- <para>Each notification has a certain priority that requires a different handling of the issue. How each priority will be dealt with is something that can be set per server. The priority mechanism is a simple system of five categories (can be more or less). </para>
- <para/>
- </section>
+<itemize>
+ <item> Central Application - server </item>
+ <item> Monitored System - client </item>
+</itemize>
- <section>
- <heading>The dataflow diagram.</heading>
- <para>The main dataflow will be as follows.</para>
- <para>
- <picture src="dataflow.png" eps="dataflow" scale="0.7"/>
- </para>
- <para/>
- </section>
-</chapter>
+<para>
+The project has been setup as a two sided system in order to be able to
+guard many computers at the same time.
+However it may be obvious that both sides of the application can very
+well be installed on a single system.
+</para>
+<para>
+To monitor a system, Gnucomo uses two kinds of input: <emph>event</emph> and
+<emph>parameters</emph>. Events occur on a system while it is running and reflect
+the transient behaviour of the system.
+Parameters reflect the current state of the system.
+The most obvious way to gather events from a monitored system is to read
+the system log files.
+Examples of events are IP packets that are rejected by the firewall or clients
+that access the http daemon.
+Parameters are obtained for example by reading configuration files or kernel
+data structures.
+Examples of parameters are the size and free space of a filesystem or
+the users that are listed in the password file.
+Both kinds of input are obtained actively or passively, i.e. by installing probe
+agents in the system which regularly aquire the system's parameters or passively
+by sending the output of programs to the Gnucomo server.
+</para>
+<para>
+When signals<footnote>A signal can be the outcome of process that finished,
+logbook entries, warnings from intrusion detection systems,
+etc. </footnote> arrive they will be stored in a file.
+When this file is delivered to a certain directory a daemon will detect
+this and start the transfer of the file. The file will be transferred
+to the central application or the client. This transfer will be triggered
+immediately after a process has finished or with a certain time-interval
+when it concerns a logbook. All output will be placed in a directory where
+the daemon detects it and ensures the transport. For transport currently
+only two mechanisms will be supported:
+</para>
+<itemize>
+ <item>
+ Encrypted file copy relying on the SSH protocol.
+ On Unix-based systems this will scp (secure copy).
+ </item>
+ <item>
+ E-Mail. The e-mail is encrypted and signed using gpg and then sent to the server.
+ Since the file format will identify the type-of-output the Subject-field of
+ the e-mail is not really needed.
+ </item>
+</itemize>
-<chapter>
+<section>
<heading>Central Application: signal handler.</heading>
<para>
+<para>
- <picture src="architecture.png" eps="architecture"/>
+ <picture src="architecture.png" eps="architecture"/>
+</para>
+<para>
Illustration 1 Basic overview of the processes on the server.
- </para>
+</para>
-On one machine signals from the network will come in. These signals can be logfiles, result files from applications, remarks entered by the administrator or whatever. Data delivery takes place into a certain directory. A daemon detects that data has come in and will enter it into the database. Once in the database stored procedures and triggers will recognize certain behavior and generate alerts. The user responsible for the server will be confronted with the alerts and can mark them, add comments to it or ignore them. Also it will be possible to do intelligent analysis on not so logical relations per computer or across computers<footnote>A good example could be a portscan on a system that takes place during a week. Normal a simple portscan takes place in a couple of minutes and will thus be easy to detect. By taking a longer period such a scan is harder to detect. </footnote>. Such scripts make detection possible, that is too time consuming to do during processing of the data. </para>
+On one machine signals from the network will come in.
+These signals can be logfiles, result files from applications,
+remarks entered by the administrator or whatever.
+Data delivery takes place into a certain directory.
+A daemon detects that data has come in and will enter it into the database.
+Once in the database stored procedures and triggers will recognize certain
+behavior and generate alerts. The user responsible for the server will be
+confronted with the alerts and can mark them, add comments to it or ignore them.
+Also it will be possible to do intelligent analysis on not so logical relations
+per computer or across computers<footnote>A good example could be a portscan
+on a system that takes place during a week. Normal a simple portscan takes
+place in a couple of minutes and will thus be easy to detect.
+By taking a longer period such a scan is harder to detect. </footnote>. Such
+scripts make detection possible, that is too time consuming to do during
+processing of the data.
+</para>
+</section>
- <section>
- <heading>Data processing.</heading>
- <para>The data processing has four tasks:</para>
- <itemize>
- <item>
- <para>Extracting data from e-mail and store it in the input-buffer. This can be done by a daemon that checks the e-mail<footnote>It would be logical to place an e-mail server like sendmail or postfix on the server. In many cases the monitored computer will be featuring a SMTP daemon. If a system is comprimised no evidence that comes through will be really trustworthy. By sending it to another machine all evidence that is available will have left the system the moment a hack takes place.</footnote> with a certain interval extracts the e-mails and leave the content in a file in the input buffer. This daemon has only rights to write to the directory it has to write to. <emph>We may as well have the email captured directly by a program with a "gnucomo: |/usr/local/bin/gnucomo-input" - like alias.</emph></para>
- </item>
- <item>
- <para>Detect files in the input buffer decrypt the content and verify the signature. Another daemon will see the log-files and starts checking if the origin is correct (by verifying the signature) and decrypting the content. The legible files will be processed by entering the data into the database.</para>
- </item>
- <item>
- <para>The database will accept the data and perform a certain number of checks as the data comes in. During the processing of the data abnormalities will be detected and entered into a notification table. The database system will also carry out more complex tasks on given moments in time and enter them as well in the notification table.</para>
- </item>
- <item>
- <para>The database system must be able to undertake action when high alerts are being entered into the database. This can be a couple of things to begin with: send an e-mail or SMS. In a later stage other technologies may be added as well. In this scheme we also make a possibility to escalate problems when no action is taken in a certain amount of time.</para>
- </item>
- </itemize>
- </section>
- <section>
- <heading>Web interface.</heading>
- <para>The web interface will used to interact with the user. The interface should be intuitive and easy to understand. More important warnings should directly draw attention. The user must be able to perform settings so that warnings can be rated differently than the original settings. </para>
- <para>The interface will do the following things:</para>
- <itemize>
- <item>
- <para>Show a list of warnings that are currently open.</para>
- </item>
- <item>
- <para>It must be possible to sort the list on all the fields shown.</para>
- </item>
- <item>
- <para>Deliver detailed information (logbook entries) upon request that have led to the warning. </para>
- </item>
- <item>
- <para>Undertake certain actions like sending <reference href="mailto:abuse@internet-provider.net">abuse@internet-provider.net</reference> e-mails with information.</para>
- </item>
- <item>
- <para>Monitor actions on outstanding issues. </para>
- </item>
- </itemize>
- <para/>
- </section>
-</chapter>
+<section>
+ <heading>Data processing.</heading>
-<chapter>
+<para>The data processing has four tasks:</para>
+<itemize>
+ <item>
+ Extracting data from e-mail and store it in the input-buffer.
+ This can be done by a daemon that checks the e-mail<footnote>It would be logical
+ to place an e-mail server like sendmail or postfix on the server.
+ In many cases the monitored computer will be featuring a SMTP daemon.
+ If a system is comprimised no evidence that comes through will be really
+ trustworthy. By sending it to another machine all evidence that is
+ available will have left the system the moment a hack takes place.</footnote>
+ with a certain interval extracts the e-mails and leave the content
+ in a file in the input buffer. This daemon has only rights to write
+ to the directory it has to write to. <emph>We may as well have the
+ email captured directly by a program with a
+ "gnucomo: |/usr/local/bin/gnucomo-input" - like alias.</emph>
+ </item>
+ <item>
+ Detect files in the input buffer decrypt the content and verify the signature.
+ Another daemon will see the log-files and starts checking if the origin is
+ correct (by verifying the signature) and decrypting the content.
+ The legible files will be processed by entering the data into the database.
+ </item>
+ <item>
+ The database will accept the data and perform a certain number of
+ checks as the data comes in. During the processing of the data
+ abnormalities will be detected and entered into a notification table.
+ The database system will also carry out more complex tasks on given
+ moments in time and enter them as well in the notification table.
+ </item>
+ <item>
+ The database system must be able to undertake action when high
+ alerts are being entered into the database.
+ This can be a couple of things to begin with:
+ send an e-mail or SMS. In a later stage other technologies may be added as well.
+ In this scheme we also make a possibility to escalate problems when no
+ action is taken in a certain amount of time.
+ </item>
+</itemize>
+
+<subsection>
+<heading>System Parameters</heading>
+
+<para>
+Gnucomo maintains the operational parameters of a monitored system for a
+number of reasons.
+The most important reason is to create notifications when somthing about a
+parameter changes while the parameter is not supposed to change.
+Such a change may be intended by the system administrator, e.g. when a
+package is upgraded, or there may be something wrong.
+In any case, you will want to know about a change in your system when it happens.
+Furthermore, a change history of a parameter's values will come in
+handy when you want to look back in time and figure out
+what happened in the past.
+Another usefull application of parameters concerns the maintenance of a
+large number of similar systems.
+When the parameters of each system are reported regularly to Gnucomo,
+deviations from the 'standard' system configuration can be easily spotted.
+</para>
+<para>
+Some properties of parameters are supposed to change regularly.
+A changed value of such a property will of course not lead to any
+notification.
+On the other hand, the change history of these parameters may provide
+interesting information about the monitored system.
+This leads to the distiction between static and dynamic properties of parameters.
+The difference between dynamic and static properties manifests itself mainly
+in the change history of the parameter's property.
+Dynamic properties typically have a change record once a day or even a couple
+of times a day.
+Change records for static properties are usually months apart.
+</para>
+<para>
+The state of parameters is scanned or probed regularly on a client system
+and reported to the Gnucomo server.
+These reports can be created in a variety of ways.
+For example, filesystems are reported with 'df', installled packages with 'rpm -qa',
+users by reading /etc/passwd, etc.
+Many other probing methods may be implemented.
+Each report from a probe holds the current value of several parameters.
+Gnucomo will check each property of these parameters against the stored knwon value.
+If the property's value changed, the actual value in the database is updated
+and a record is added to the change history of the parameter.
+</para>
+</subsection>
+</section>
+
+<section>
+ <heading>Web interface</heading>
+
+<para>
+The web interface will used to interact with the user.
+The interface should be intuitive and easy to understand.
+More important warnings should directly draw attention.
+The user must be able to perform settings so that warnings
+can be rated differently than the original settings.
+</para>
+<para>
+The interface will do the following things:
+</para>
+<itemize>
+ <item>
+ Show a list of warnings that are currently open.
+ </item>
+ <item>
+ It must be possible to sort the list on all the fields shown.
+ </item>
+ <item>
+ Deliver detailed information (logbook entries) upon request that have led to the warning.
+ </item>
+ <item>
+ Undertake certain actions like sending
+ <reference href="mailto:abuse@internet-provider.net">abuse@internet-provider.net</reference>
+ e-mails with information.
+ </item>
+ <item>
+ Monitor actions on outstanding issues.
+ </item>
+</itemize>
+<para/>
+</section>
+
+<section>
+ <heading>Priority mechanism.</heading>
+<para>
+Each notification has a certain priority that requires a different handling
+of the issue. How each priority will be dealt with is something that can
+be set per server. The priority mechanism is a simple system of five
+categories (can be more or less).
+</para>
+</section>
+
+<section>
+ <heading>The dataflow diagram.</heading>
+<para>The main dataflow will be as follows.</para>
+<para>
+ <picture src="dataflow.png" eps="dataflow" scale="0.7"/>
+</para>
+<para/>
+</section>
+
+<section>
<heading>Sending messages to the central gnucomo system.</heading>
-<para>One of the main tasks is getting all the messages to the database. Ultimately gnucomo will support multiple ways of receiving the data. Basically anything goes, but two mechanisms will be supported in the project to begin with:</para>
+<para>
+One of the main tasks is getting all the messages to the database.
+Ultimately gnucomo will support multiple ways of receiving the data.
+Basically anything goes, but two mechanisms will be supported in the project
+to begin with:
+</para>
<enumerate>
<item>
- <para>E-mail. Messages and elements from logfiles will be sent through e-mail.</para>
+ E-mail. Messages and elements from logfiles will be sent through e-mail.
</item>
<item>
- <para>File copy. Using technologies like scp or ftp (not preferred due to the insecure nature) can place files directly in the receiving directory. </para>
+ File copy. Using technologies like scp or ftp (not preferred due to
+ the insecure nature) can place files directly in the receiving directory.
</item>
</enumerate>
- <section>
- <heading>Ensuring data integrity.</heading>
-
<para><TO BE DESCRIBED></para>
- </section>
+<subsection>
+ <heading>Ensuring data integrity.</heading>
+<para><TO BE DESCRIBED></para>
+</subsection>
- <section>
- <heading>Directories and filenames on the server.</heading>
+<subsection>
+ <heading>Directories and filenames on the server.</heading>
<para>
Files will be dealt with as if gnucomo were a user (actually there will
<para>Urgency</para>
</col>
<col>
- <para>This indicates the urgency of the file. The lower the number the higher it will rank when an overview is given. Standard files are ranked value 3. The ranking works as follows:</para>
- <para>* 1 <strong>Urgent flash message.</strong> Something urgent needs to be reported. This is only used for emergencies like serious alarms.</para>
- <para>* 2 <strong>Rapid delivery.</strong> An important message has to get through that is more important than normal delivery, but is not top priority like an emergency.</para>
- <para>* 3 <strong>Normal.</strong> This is used in most case for normal messages.</para>
- <para>* 4 <strong>Low priority.</strong> This is data would be useful to place into the system as nice to have.</para>
+ <para>This indicates the urgency of the file. The lower the number the higher
+ it will rank when an overview is given. Standard files are ranked value 3.
+ The ranking works as follows:</para>
+ <para>* 1 <strong>Urgent flash message.</strong> Something urgent needs to
+ be reported. This is only used for emergencies like serious alarms.</para>
+ <para>* 2 <strong>Rapid delivery.</strong> An important message has to get
+ through that is more important than normal delivery, but is not top
+ priority like an emergency.</para>
+ <para>* 3 <strong>Normal.</strong> This is used in most
+ case for normal messages.</para>
+ <para>* 4 <strong>Low priority.</strong> This is data would be useful
+ to place into the system as nice to have.</para>
</col>
</row>
<row>
<para>TypeOfMessage</para>
</col>
<col>
- <para>This is an indicator what type of data is delivered to gnucomo. There are several categories:</para>
+ <para>This is an indicator what type of data is delivered to gnucomo.
+ There are several categories:</para>
<para>* <strong>cron.</strong> This data comes from the /var/log/cron-log (unix).</para>
<para>* <strong>httpaccess.</strong> This data comes from the normal http-log (Apache).</para>
<para>* <strong>httperror.</strong> This data comes from the http_error-log (Apache).</para>
<para>* <strong>maillog.</strong> This data comes from the /var/log/maillog (unix).</para>
- <para>* <strong>messages.</strong> The data delivered here comes from the /var/log/messages file (unix)</para>
- <para>* <strong>text.</strong> This file contains a message in plaintext generated by a gnucomo-client and can be anything. It will be dealt with as plaintext.</para>
+ <para>* <strong>messages.</strong> The data delivered here comes from
+ the /var/log/messages file (unix)</para>
+ <para>* <strong>text.</strong> This file contains a message in plaintext
+ generated by a gnucomo-client and can be anything.
+ It will be dealt with as plaintext.</para>
</col>
</row>
<row>
<para>Timestamp</para>
</col>
<col>
- <para>The timestamp is made in war-log (YYYYMMDDHHMMSS) format. The timestamp is generated on the client in GMT (to discover discrepancies in timing).</para>
+ <para>The timestamp is made in war-log (YYYYMMDDHHMMSS) format.
+ The timestamp is generated on the client in GMT (to discover
+ discrepancies in timing).</para>
</col>
</row>
<row>
<para>Objectid</para>
</col>
<col>
- <para>The objectid is the id that is used within the central gnucomo system to recognize the client. Based on this entry the database can link the data to the correct object. Also gpg can obtain the correct e-mail address (by running a query in the database) for verification of the signature of the crypted message.</para>
+ <para>The objectid is the id that is used within the central
+ gnucomo system to recognize the client.
+ Based on this entry the database can link the data to the correct object.
+ Also gpg can obtain the correct e-mail address (by running a query
+ in the database) for verification of the signature of the crypted message.</para>
</col>
</row>
<row>
<para>Indicates the file type:</para>
<para>* <strong>asc.</strong> Plaintext ASCII</para>
<para>* <strong>gpg.</strong> gpg-crypted data.</para>
- <para>* <strong>und.</strong> Undertermined data. When files come in by e-mail it is not 100% sure if they are crypted or not. These data has first to be analyzed before it is moved to the correct queue. </para>
+ <para>* <strong>und.</strong> Undertermined data. When files
+ come in by e-mail it is not 100% sure if they are crypted
+ or not. These data has first to be analyzed before it is
+ moved to the correct queue. </para>
</col>
</row>
</table>
-<subsection>
+<subsubsection>
<heading>Directory for incoming data.</heading>
-<para>For incoming messages there will be separate directories (<code>/home/gnucomo/incoming/</code>)for:</para>
+<para>
+For incoming messages there will be separate directories
+(<code>/home/gnucomo/incoming/</code>)for:
+</para>
<enumerate>
<item>
- <para>Dropbox. This directory is ready to receive data from all sorts of systems. This directory is world writeable (<strong><emph>but not deleteable!</emph></strong>): <code>/home/gnucomo/incoming/dropbox/</code></para>
+ Dropbox. This directory is ready to receive data from all sorts of systems.
+ This directory is world writeable (<strong><emph>but not deleteable!</emph></strong>):
+ <code>/home/gnucomo/incoming/dropbox/</code>
</item>
<item>
- <para>Encrypted messages. In this directory messages will be placed that are still gpg-crypted. In this directory the files await decryption and verification of the signature. This directory is: <code>/home/gnucomo/incoming/crypted/</code></para>
+ Encrypted messages. In this directory messages will be placed that are still
+ gpg-crypted. In this directory the files await decryption and verification of
+ the signature. This directory is: <code>/home/gnucomo/incoming/crypted/</code>
</item>
<item>
- <para>Decrypted with errors. During the decryption exercise anything can go wrong. Decrypting can fail or the signature may show errors. If this happens the original message is moved to a different directory: <code>/home/gnucomo/incoming/cryptfail/</code></para>
+ Decrypted with errors. During the decryption exercise anything can go wrong.
+ Decrypting can fail or the signature may show errors. If this happens the
+ original message is moved to a different directory:
+ <code>/home/gnucomo/incoming/cryptfail/</code>
</item>
<item>
- <para>Inbox. After successful decryption the decrypted message is moved to the inbox. Data that is not encrypted can be moved here from the dropbox after certain verifications. The directory is: <code>/home/gnucomo/incoming/inbox/</code></para>
+ Inbox. After successful decryption the decrypted message is moved to the inbox.
+ Data that is not encrypted can be moved here from the dropbox after certain
+ verifications. The directory is: <code>/home/gnucomo/incoming/inbox/</code>
</item>
<item>
- <para>Processed. After processing the data the file is stored in the directory processed for further reference for a certain amount of time. The directory is: <code>/home/gnucomo/incoming/processed/</code></para>
+ Processed. After processing the data the file is stored in the directory
+ processed for further reference for a certain amount of time.
+ The directory is: <code>/home/gnucomo/incoming/processed/</code>
</item>
<item>
- <para>Archive. After a certain period the files will be archived. Since not every system will archive everything this directory may also be a symbolic link to <code>/dev/null</code>. The used directory is: <code>/home/gnucomo/archive/</code></para>
+ Archive. After a certain period the files will be archived.
+ Since not every system will archive everything this directory may
+ also be a symbolic link to <code>/dev/null</code>.
+ The used directory is: <code>/home/gnucomo/archive/</code>
</item>
</enumerate>
-</subsection>
+</subsubsection>
-<subsection>
+<subsubsection>
<heading>Directory for outgoing data.</heading>
- <para>For outgoing messages the directory <code>/home/gnucomo/outgoing</code> will be used. This directory knows a couple of sub directories:</para>
+<para>
+For outgoing messages the directory <code>/home/gnucomo/outgoing</code> will
+be used. This directory knows a couple of sub directories:
+</para>
<enumerate>
<item>
- <para>Dropbox. This directory is used by the central gnucomo system to place outgoing messages in. The used directory is: <code>/home/gnucomo/outgoing/dropbox/</code></para>
+ Dropbox. This directory is used by the central gnucomo system to
+ place outgoing messages in.
+ The used directory is: <code>/home/gnucomo/outgoing/dropbox/</code>
</item>
<item>
- <para>Outbox. After processing the message (including signing and encrypting when applicable) the messages are placed in the outbox: <code>/home/gnucomo/outgoing/outbox/</code></para>
+ Outbox. After processing the message (including signing and encrypting
+ when applicable) the messages are placed in the outbox:
+ <code>/home/gnucomo/outgoing/outbox/</code>
</item>
<item>
- <para>Processed. After processing the data the file is stored in the directory processed for further reference for a certain amount of time. After sending a message a confirmation will be made that is saved as an incoming message. The directory is: <code>/home/gnucomo/outgoing/processed/</code></para>
+ Processed. After processing the data the file is stored in the
+ directory processed for further reference for a certain amount
+ of time. After sending a message a confirmation will be made that
+ is saved as an incoming message.
+ The directory is: <code>/home/gnucomo/outgoing/processed/</code>
</item>
<item>
- <para>Archive. After a certain period the files will be archived. Since not every system will archive everything this directory may also be a symbolic link to <code>/dev/null</code>. The used directory is: <code>/home/gnucomo/archive/</code></para>
+ Archive. After a certain period the files will be archived.
+ Since not every system will archive everything this directory
+ may also be a symbolic link to <code>/dev/null</code>.
+ The used directory is: <code>/home/gnucomo/archive/</code>
</item>
</enumerate>
-</subsection>
+</subsubsection>
-<subsection>
+<subsubsection>
<heading>Overview. </heading>
<para>The total directory-structure looks like this:</para>
/home/gnucomo/outgoing/processed
</verbatim>
+</subsubsection>
</subsection>
-</section>
-<section>
+<subsection>
<heading>Directories on the client-side.</heading>
-<para>On the client-side the files that need to be transmitted will be placed in a directory system as well. In the future this system may not be in use at all devices (routers, certain MS Windows-machine, IP Telephones, etc.). For those systems a different mechanism will become be described here. Initially we focus on Linux systems that will enter data into the database.</para>
- <para>The filename convention will be totally identical to the filename convention on the server, since this the same mechanism.</para>
- <para>To facilitate gnucomo client and server on one and the same machine the gnucomo-client should have a different default user. For this purpose the user <strong>gcm_client</strong> will be created. </para>
+<para>
+On the client-side the files that need to be transmitted will be placed in a
+directory system as well. In the future this system may not be in use at all
+devices (routers, certain MS Windows-machine, IP Telephones, etc.). For those
+systems a different mechanism will become be described here.
+Initially we focus on Linux systems that will enter data into the database.
+</para>
+<para>
+The filename convention will be totally identical to the filename
+convention on the server, since this the same mechanism.
+</para>
+<para>
+To facilitate gnucomo client and server on one and the same machine
+the gnucomo-client should have a different default user.
+For this purpose the user <strong>gcm_client</strong> will be created.
+</para>
-<subsection>
+<subsubsection>
<heading>Directory for incoming data.</heading>
- <para>For incoming messages there will be separate directories (<code>/home/gnucomo/incoming/</code>)for:</para>
+<para>
+For incoming messages there will be separate directories
+(<code>/home/gnucomo/incoming/</code>)for:
+</para>
<enumerate>
<item>
- <para>Dropbox. This directory is ready to receive data from all sorts of systems. This directory is world writeable (<emph><strong>but not deleteable!</strong></emph>): <code>/home/gcm_client/incoming/dropbox/</code></para>
+ Dropbox. This directory is ready to receive data from all sorts of systems.
+ This directory is world writeable (<emph><strong>but not deleteable!</strong></emph>):
+ <code>/home/gcm_client/incoming/dropbox/</code>
</item>
<item>
- <para>Encrypted messages. In this directory messages will be placed that are still gpg-crypted. In this directory the files await decryption and verification of the signature. This directory is: <code>/home/gcm_client/incoming/crypted/</code></para>
+ Encrypted messages. In this directory messages will be placed that
+ are still gpg-crypted. In this directory the files await decryption
+ and verification of the signature.
+ This directory is: <code>/home/gcm_client/incoming/crypted/</code>
</item>
<item>
- <para>Decrypted with errors. During the decryption exercise anything can go wrong. Decrypting can fail or the signature may show errors. If this happens the original message is moved to a different directory: <code>/home/gcm_client/incoming/cryptfail/</code></para>
+ Decrypted with errors. During the decryption exercise anything can go wrong.
+ Decrypting can fail or the signature may show errors.
+ If this happens the original message is moved to a different directory:
+ <code>/home/gcm_client/incoming/cryptfail/</code>
</item>
<item>
- <para>Inbox. After successful decryption the decrypted message is moved to the inbox. Data that is not encrypted can be moved here from the dropbox after certain verifications. The directory is: <code>/home/gcm_client/incoming/inbox/</code></para>
+ Inbox. After successful decryption the decrypted message is moved
+ to the inbox. Data that is not encrypted can be moved here from
+ the dropbox after certain verifications. The directory is:
+ <code>/home/gcm_client/incoming/inbox/</code>
</item>
<item>
- <para>Processed. After processing the data the file is stored in the directory processed for further reference for a certain amount of time. The directory is: <code>/home/gcm_client/incoming/processed/</code></para>
+ Processed. After processing the data the file is stored
+ in the directory processed for further reference for a
+ certain amount of time. The directory is:
+ <code>/home/gcm_client/incoming/processed/</code>
</item>
</enumerate>
-</subsection>
+</subsubsection>
-<subsection>
+<subsubsection>
<heading>Directory for outgoing data.</heading>
- <para>For outgoing messages the directory <code>/home/gcm_client/outgoing</code> will be used. This directory knows a couple of sub directories:</para>
+<para>
+For outgoing messages the directory <code>/home/gcm_client/outgoing</code> will be used.
+This directory knows a couple of sub directories:
+</para>
<enumerate>
<item>
- <para>Dropbox. This directory is used by the central gnucomo system to place outgoing messages in. The used directory is: <code>/home/gcm_client/outgoing/dropbox/</code></para>
+ Dropbox. This directory is used by the central gnucomo system to place
+ outgoing messages in.
+ The used directory is: <code>/home/gcm_client/outgoing/dropbox/</code>
</item>
<item>
- <para>Outbox. After processing the message (including signing and encrypting when applicable) the messages are placed in the outbox: <code>/home/gcm_client/outgoing/outbox/</code></para>
+ Outbox. After processing the message (including signing and
+ encrypting when applicable) the messages are placed in the outbox:
+ <code>/home/gcm_client/outgoing/outbox/</code>
</item>
<item>
- <para>Processed. After processing the data the file is stored in the directory processed for further reference for a certain amount of time. After sending a message a confirmation will be made that is saved as an incoming message. The directory is: <code>/home/gcm_client/outgoing/processed/</code></para>
+ Processed. After processing the data the file is stored in the
+ directory processed for further reference for a certain amount
+ of time. After sending a message a confirmation will be made
+ that is saved as an incoming message.
+ The directory is: <code>/home/gcm_client/outgoing/processed/</code>
</item>
</enumerate>
-</subsection>
+</subsubsection>
-<subsection>
+<subsubsection>
<heading>Overview. </heading>
<para>The total directory-structure looks like this:</para>
/home/gcm_client/outgoing/outbox
/home/gcm_client/outgoing/processed
</verbatim>
+</subsubsection>
</subsection>
</section>
-</chapter>
-<chapter>
+<section>
<heading>Getting data into the database.</heading>
- <para>The files in the /home/gnucomo/incoming/inbox/ should be stored in the database. For this purpose there is a table <emph>unprocessed_log</emph>. The data of the filename as well as the content of the file need to be placed in one record.</para>
- <para>There are some fields that have to be addressed immediately:</para>
+
+<para>
+The files in the /home/gnucomo/incoming/inbox/ should be stored in the database.
+For this purpose there is a table <emph>unprocessed_log</emph>.
+The data of the filename as well as the content of the file need
+to be placed in one record.
+</para>
+<para>
+There are some fields that have to be addressed immediately:
+</para>
<enumerate>
<item>
- <para>Servicecode: The servicecode will be obtained out of the filename the is the type_of_message-field in the filename. </para>
+ Servicecode: The servicecode will be obtained out of the filename the
+ is the type_of_message-field in the filename.
</item>
<item>
- <para>Objectid: This data is given in the filename it can be found in the <emph>objectid</emph>-part of the filename.</para>
+ Objectid: This data is given in the filename it can be found
+ in the <emph>objectid</emph>-part of the filename.
</item>
<item>
- <para>Logdata: The data is saved as follows: filename (unaltered) <CR>textual data of the file.</para>
+ Logdata: The data is saved as follows: filename (unaltered)
+ <CR>textual data of the file.
</item>
</enumerate>
- <para>The daemon application that delivers the data is called <strong>gcm-input</strong>. It performs the following steps with no extra functionality:</para>
+<para>
+The daemon application that delivers the data is called <strong>gcm-input</strong>.
+It performs the following steps with no extra functionality:
+</para>
<enumerate>
<item>
</item>
</enumerate>
- <para>To write the data in the database a database user gcm_input exists. This user has only the right to enter data into the database. There are no deletion, update or select-permissions.</para>
- <para/>
+<para>
+To write the data in the database a database user gcm_input exists.
+This user has only the right to enter data into the database.
+There are no deletion, update or select-permissions.
+</para>
+
+</section>
</chapter>
<chapter>
- <heading>The database.</heading>
+ <heading><label name='database'/>The database.</heading>
- <para>The database is the heart of the system. It will contain all event-data of multiple computers. The intelligence that can be performed on the database will be placed there. To do this as integrated as possible stored procedures and triggers will be used. To begin with we have selected checks to be performed that will be expanded throughout time.</para>
- <para>Since the gnucomo database and files contain sensitive data security measures have to be in place. Several database users will be exist that have limited rights to perform a certain task ensuring some protection against unauthorized access. However these mechanisms on it's own will work fine, bad maintenance may still screw-up good security. Good database maintenance is needed. For the gnucomo the protection of the valid authentic nature of the data in the database has our highest priority.</para>
- <para>A <emph>table-name</emph> in this chapter is written in cursive writing.</para>
+<para>
+The database is the heart of the system.
+It will contain all event-data of multiple computers.
+The intelligence that can be performed on the database will be placed there.
+To do this as integratedly as possible stored procedures and triggers will be used.
+To begin with we have selected checks to be performed that will be
+expanded throughout time.
+</para>
+<para>
+Since the gnucomo database and files contain sensitive data
+security measures have to be in place. Several database users
+will exist that have limited rights to perform a certain task
+ensuring some protection against unauthorized access.
+However these mechanisms on it's own will work fine, bad maintenance may still
+screw-up good security. Good database maintenance is needed.
+For the gnucomo the protection of the valid authentic nature of
+the data in the database has our highest priority.
+</para>
+<para>
+A <emph>table-name</emph> in this chapter is written in cursive writing.
+</para>
<section>
<heading>Database conventions.</heading>
- <para>The database will be built according the following conventions:</para>
- <itemize>
- <item>
- <para>All names for tables, indexes and fields will be written in lower case and will be singular. Spaces in names are not allowed.</para>
- </item>
- <item>
- <para>If a table contains field referring to other tables the mother-tables are mentioned in alphabetical order with an underscore between the table-names. So <emph>object</emph> and <emph>user</emph> will make a third table <emph>object_user</emph></para>
- </item>
- <item>
- <para>A data access user (being the interface) is not allowed to write to the log-entries and the warnings. To change the state of a warning a stored procedure will do so by having a different entry in a table. This should make it possible to discover who did what at which moment in time.</para>
- </item>
- <item>
- <para>The name of two related fields that make the relationship between two tables will be the same to avoid confusion.</para>
- </item>
- </itemize>
+
+<para>
+The database will be built according the following conventions:
+</para>
+<itemize>
+ <item>
+ All names for tables, indexes and fields will be written in lower
+ case and will be singular. Spaces in names are not allowed.
+ </item>
+ <item>
+ If a table contains field referring to other tables the mother-tables are
+ mentioned in alphabetical order with an underscore between the table-names.
+ So <emph>object</emph> and <emph>user</emph>
+ will make a third table <emph>object_user</emph>
+ </item>
+ <item>
+ A data access user (being the interface) is not allowed to write to
+ the log-entries and the warnings. To change the state of a warning
+ a stored procedure will do so by having a different entry in a table.
+ This should make it possible to discover who did what at which moment in time.
+ </item>
+ <item>
+ The name of two related fields that make the relationship between two
+ tables will be the same to avoid confusion.
+ </item>
+</itemize>
</section>
<section>
<heading>Database design.</heading>
- <para>In the design we anticipate to deliver an as best as possible database performance. That means that data that needs to be entered occasionally can be heavily indexed to increase performance. However data that is mainly stored will only be indexed marginally to have the best possible performance on data entry. If during one of the checks on data-entry a notification is made, the information related to that notification will be indexed very well to increase retrieval performance. What we will try to avoid is that the user interface will cause full table scan and affect the performance of the overall system dramatically. One of the techniques to increase performance on display is to work with views. So where it is feasible we will use them.</para>
- <para>In general the database must also be maintained well. So daily maintenance scripts should keep the performance good<footnote>PostgreSQL seems to have very good features to do proper maintenance and they have to be exploited to the full extend.</footnote>. </para>
+<para>
+In the design we anticipate to deliver an as best as possible database performance.
+That means that data that needs to be entered occasionally can be heavily indexed
+to increase performance. However data that is mainly stored will only be indexed
+marginally to have the best possible performance on data entry. If during one of
+the checks on data-entry a notification is made, the information related to that
+notification will be indexed very well to increase retrieval performance.
+What we will try to avoid is that the user interface will cause full table
+scan and affect the performance of the overall system dramatically.
+One of the techniques to increase performance on display is to work with views.
+So where it is feasible we will use them.
+</para>
- <subsection>
- <heading>Actual design.</heading>
+<para>
+The following model pictures the database as described in the remainder
+of this chapter.
+</para>
+<para>
+ <picture src="erd.png" eps="erd" scale="0.7"/>
+</para>
- <para>In this part of the chapter the tables will be explained and then described with all important elements. Per table a sub-chapter will be created. Each table will have a table design, indexes, relationships and the required data (for those tables where the data itself is relevant in the design or sample data (for those cases where no set data is needed). For the relationships beside a description the subset of the total schema has been incorporated in the document so that it is more clear what exactly is meant. Due to the complex nature of the design those drawings sometimes will seem funny.</para>
- <para>For the fieldtypes the types of PostgreSQL will be used. These values can be found in Chapter 3 of the PostgreSQL User Manual (<reference href="http://www.postgresql.org/">http://www.postgresql.org</reference>). </para>
- <para>For indexes primary keys are always <emph>unique</emph> and called primary key in the name. Since unique indexes within PostgreSQL only work on B-Tree indexes (which is default) we will use B-Tree for all indexes. In cases where an exception is made the used type of index will be indicated in the characteristics. A footnote will explain why a different type of index has been selected.</para>
+<para>
+In general the database must also be maintained well.
+So daily maintenance scripts should keep the performance good<footnote>PostgreSQL seems
+to have very good features to do proper maintenance and they have to be exploited to
+the full extend.</footnote>.
+</para>
-<subsubsection>
+</section>
+<section>
+ <heading>Actual design.</heading>
+
+<para>
+In this part of the chapter the tables will be explained and then described
+with all important elements. Per table a sub-chapter will be created.
+Each table will have a table design, indexes, relationships and the required data
+(for those tables where the data itself is relevant in the design or sample data
+(for those cases where no set data is needed). For the relationships beside a
+description the subset of the total schema has been incorporated in
+the document so that it is more clear what exactly is meant.
+Due to the complex nature of the design those drawings sometimes will seem funny.
+</para>
+<para>
+For the fieldtypes the types of PostgreSQL will be used.
+These values can be found in Chapter 3 of the PostgreSQL User Manual
+(<reference href="http://www.postgresql.org/">http://www.postgresql.org</reference>).
+</para>
+<para>
+For indexes primary keys are always <emph>unique</emph> and called
+primary key in the name. Since unique indexes within PostgreSQL only
+work on B-Tree indexes (which is default) we will use B-Tree for all indexes.
+In cases where an exception is made the used type of index will be indicated
+in the characteristics. A footnote will explain why a different type of
+index has been selected.
+</para>
+
+<subsection>
<heading>action</heading>
<para>
-In the table action all recognized actions that can be taken are stored. Several actions will lead to change in <emph>statuscode</emph>. However this doesn't apply to all actions<footnote>A status NEW of a notification that cannot be dealt with automatically will <strong>not</strong> change before a user looked at it.</footnote>. Actions take place through background processes and the interface. This table mainly is used for retrieval so indexing will be done as much as possible.
+This table stores the actions that are recognized by the system.
+Whenever a situation in the database has been raised (through the table notification)
+one expects steps to be taken to resolve these issues.
+All these steps are stored in this table.
+Each step beginning with detection until a case is closed can be traced back through this table.
+This will support the <strong>accountability</strong> (who is responsible for which action)
+and <strong>non-repudiation</strong> (being able to trace what exactly happened to each notification).
+In the table action all recognized actions that can be taken are stored.
+Several of the taken actions will lead to change in <emph>statuscode</emph>.
+When this is the case either the server-side of the interfaces or the gcm_daemon will perform this task.
+This doesn't apply to all actions though.<footnote>A status NEW
+of a notification that cannot be dealt with automatically will
+<strong>not</strong> change before a user looked at it.</footnote>.
+Actions take place through background processes and the interface.
+Each action known by the system will be delivered by the software.
+This table mainly is used for retrieval so indexing will be done as much as possible.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
<row>
<col> actionid </col> <col> Bigserial </col> <col> 8 </col>
<col>
- Autonumber bigint (eight bit)
+ Autonumber bigint (eight bit). Unique identifyer to refer to when this action is taken.
</col>
</row>
<row>
<col> actionname </col> <col> Text </col> <col> <para/> </col>
<col>
- Short descriptive name for the type of action
+ Short descriptive name for the type of action.
</col>
</row>
<row>
<col> statuscode </col> <col> Varchar </col> <col> 3 </col>
<col>
New status that will be given to a notification when this action takes place.
+ If the action occurs the status in the notification can change because of this.
+ The statuscode will indicate what the new status has to be.
+ If this field is left empty no change to the status will occur.
</col>
</row>
<row>
</table>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
</table>
<para/>
-</paragraph>
-<paragraph>
+</subsubsection>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<row>
<col> actionid </col> <col> action_notification_user </col>
<col>
- Each action that takes place will be log. The Actionid will bring
- classification to the individual records. This relationship has to be enforced.
+ Each action that takes placed will be stored. The Actionid will bring
+ classification to the individual records (that reflect what it means if the action-occurred).
+ This relationship has to be enforced.
</col>
</row>
</table>
<para>
<picture src="erd-action.png" eps="erd-action"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Default data in table.</heading>
<para>
The user cannot change this or add value to it.
</para>
-<table cpos='llp{4cm}p{6cm}'>
+<table cpos='p{1cm}p{2cm}p{1.5cm}p{6cm}'>
<thead>
- <col> Actionid </col> <col> Actionname </col> <col> Statuscode </col>
+ <col>Actionid</col><col>Actionname</col><col>Statuscode</col>
<col>
Description
</col>
</thead>
<row>
- <col> 1 </col> <col> Entry in the system </col> <col> NEW </col>
+ <col>1</col><col>Entry in the system</col><col>NEW</col>
<col>
This indicates that a notification is entered into the system. The status is a <strong>NEW</strong>.
</col>
</row>
<row>
- <col> 2 </col> <col> Displayed to user </col> <col> OPN </col>
+ <col>2</col><col>Displayed to user</col><col>OPN</col>
<col>
- The notification has been displayed to the user. It is not guaranteed that the user has read the notification, but he/she should be aware of it. The status will now be changed to <strong>OPEN</strong> if the current status is <strong>NEW</strong>.
+ The notification has been displayed to the user.
+ It is not guaranteed that the user has read the notification,
+ but he/she should be aware of it.
+ The status will now be changed to <strong>OPEN</strong> if the current
+ status is <strong>NEW</strong>.
</col>
</row>
<row>
- <col> 3 </col> <col> Remarks added </col> <col> PEN </col>
+ <col>3</col><col>Remarks added</col><col>PEN</col>
<col>
- Remarks have been added to the notification. The status has now been changed to <strong>PENDING</strong>.
+ Remarks have been added to the notification. The status has now been
+ changed to <strong>PENDING</strong>.
</col>
</row>
<row>
- <col> 4 </col> <col> Priority changed manually </col> <col> PEN </col>
+ <col>4</col><col>Priority changed manually</col><col>PEN</col>
<col>
- The priority of the notification has been changed by the user. The new status is now <strong>PENDING</strong>.
+ The priority of the notification has been changed by the user.
+ The new status is now <strong>PENDING</strong>.
</col>
</row>
<row>
- <col> 5 </col> <col> Priority changed automatically </col> <col> PEN </col>
+ <col>5</col><col>Priority changed automatically</col><col>PEN</col>
<col>
- The priority of the notification has been changed by the system. If the status is not <strong>OPEN</strong> or <strong>NEW</strong> the new status is become <strong>PENDING</strong>.
+ The priority of the notification has been changed by the system.
+ If the status is not <strong>OPEN</strong> or <strong>NEW</strong> the new
+ status is become <strong>PENDING</strong>.
</col>
</row>
<row>
- <col> 6 </col> <col> Action taken </col> <col> PEN </col>
+ <col>6</col><col>Action taken</col><col>PEN</col>
<col>
A action has been taken. The status is now <strong>PENDING</strong>.
</col>
</row>
<row>
- <col> 7 </col> <col> Assignment to user </col> <col> PEN </col>
+ <col>7</col><col>Assignment to user</col><col>PEN</col>
<col>
- A notification has been explicitly assigned to another user. The status is now <strong>PENDING</strong>.
+ A notification has been explicitly assigned to another user.
+ The status is now <strong>PENDING</strong>.
</col>
</row>
<row>
- <col> 8 </col> <col> More information or research needed. </col> <col> INV </col>
+ <col>8</col><col>More information or research needed.</col><col>INV</col>
<col>
- The notification is relevant and will be handled, however more information or research will be needed. The status is <strong>UNDER INVESTIGATION</strong>.
+ The notification is relevant and will be handled, however more information
+ or research will be needed. The status is <strong>UNDER INVESTIGATION</strong>.
</col>
</row>
<row>
- <col> 9 </col> <col> Make output reference. </col> <col> REF </col>
+ <col>9</col><col>Make output reference.</col><col>REF</col>
<col>
- Automated output from an object has been sent to gnucomo. The input has been identified as a valid reference for future. Status is now <strong>REFERENCE</strong><footnote>A reference is used to find differences in output. This feature must reduce the number of wrongful alerts. Only if a change has taken place a notification is generated. First time reports will always generate a notification. By signing this off the system will be silent again.</footnote>.
+ Automated output from an object has been sent to gnucomo.
+ The input has been identified as a valid reference for future.
+ Status is now <strong>REFERENCE</strong><footnote>A reference is used to
+ find differences in output. This feature must reduce the number of wrongful alerts.
+ Only if a change has taken place a notification is generated.
+ First time reports will always generate a notification.
+ By signing this off the system will be silent again.</footnote>.
</col>
</row>
<row>
- <col> 10 </col> <col> Job output no longer reference. </col> <col> CLS </col>
+ <col>10</col><col>Job output no longer reference.</col><col>CLS</col>
<col>
- By making a newer job output reference this output has been obsoleted. Since once it was a reference the notification can be closed. The new status for the notification is now <strong>CLOSED</strong>.
+ By making a newer job output reference this output has been obsoleted.
+ Since once it was a reference the notification can be closed.
+ The new status for the notification is now <strong>CLOSED</strong>.
</col>
</row>
<row>
- <col> 11 </col> <col> Action taken please verify. </col> <col> VRF </col>
+ <col>11</col><col>Action taken please verify.</col><col>VRF</col>
<col>
- An action has been taken and things should have been resolved. Before the notification can be closed a verification has to be done. New status is now <strong>VERIFY</strong>.
+ An action has been taken and things should have been resolved.
+ Before the notification can be closed a verification has to be done.
+ New status is now <strong>VERIFY</strong>.
</col>
</row>
<row>
- <col> 12 </col> <col> Action not accepted. </col> <col> PEN </col>
+ <col>12</col><col>Action not accepted.</col><col>PEN</col>
<col>
- A check has been done and the results were not good. New verification is needed. New status is now <strong>PENDING</strong>.
+ A check has been done and the results were not good.
+ New verification is needed.
+ New status is now <strong>PENDING</strong>.
</col>
</row>
<row>
- <col> 13 </col> <col> Action verified </col> <col> CLS </col>
+ <col>13</col><col>Action verified</col><col>CLS</col>
<col>
- The verification for the action has been done and the action is approved. The new status is now <strong>CLOSED</strong>.
+ The verification for the action has been done and the action is approved.
+ The new status is now <strong>CLOSED</strong>.
</col>
</row>
<row>
- <col> 14 </col> <col> E-mail sent </col> <col> OPN<footnote>Only if the status is <strong>NEW</strong>.</footnote> </col>
+ <col>14</col><col>E-mail sent</col>
+ <col>OPN<footnote>Only if the status is <strong>NEW</strong>.</footnote></col>
<col>
An e-mail has been sent.
</col>
</row>
<row>
- <col> 15 </col> <col> SMS sent </col> <col> OPN<footnote>Only if the status is <strong>NEW</strong>.</footnote> </col>
+ <col>15</col><col> SMS sent </col>
+ <col>OPN<footnote>Only if the status is <strong>NEW</strong>.</footnote></col>
<col>
A SMS has been sent.
</col>
</row>
<row>
- <col> 16 </col> <col> Fax sent </col> <col> OPN<footnote>Only if the status is <strong>NEW</strong>.</footnote> </col>
+ <col>16</col><col>Fax sent</col>
+ <col>OPN<footnote>Only if the status is <strong>NEW</strong>.</footnote></col>
<col>
A fax has been sent.
</col>
</row>
<row>
- <col> 17 </col> <col> Log-entries shown </col> <col> XXXX </col>
+ <col>17</col><col>Log-entries shown</col><col>XXXX</col>
<col>
The log entries have been shown. No changes to the status made.
</col>
</row>
<row>
- <col> 18 </col> <col> Notification closed </col> <col> CLS </col>
+ <col>18</col><col>Notification closed</col><col>CLS</col>
<col>
Notification has been closed.
</col>
</row>
<row>
- <col> 19 </col> <col> Notification reopened </col> <col> OPN </col>
+ <col>19</col><col>Notification reopened</col><col>OPN</col>
<col>
Notification has been re-opened
</col>
</row>
</table>
-</paragraph>
-
</subsubsection>
-<subsubsection>
- <heading>Action_notification_user.</heading>
+</subsection>
+
+<subsection>
+ <heading>Action_user.</heading>
<para>
-In the table action_notification_user each step that is taken regarding a
-notification is logged. This table is very important for later use if
+This table stores each actual action taken.
+Where the action basically stores the type of actions that can be taken,
+this table says 'at this point in time for this notification this action was taken'.
+Using this table it is traceable who did what at which moment in time.
+The table is very important for later use if
something goes wrong, but is also relevant for the interface.
-All steps of a notification can be traced here.
+All steps of a in the process of a notification can be traced using this table.
There will be a lot of entries here, but retrieval is more crucial for
performance than data entry. So indexing on logic fields is very relevant.
Processing might be slower, but that's worth the price.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
<row>
<col> actionstepid </col> <col> Bigserial </col> <col> 8 </col>
<col>
- Autonumber bigint (eight bit)
+ Autonumber bigint (eight bit). A unique identifyer to indicate each step in the process.
</col>
</row>
<row>
<col> actionid </col> <col> Bigint </col> <col> 8 </col>
<col>
- Reference to the action that is being registered here.
+ Reference to the action that is being registered here. This field refers to the actual action taken.
</col>
</row>
<row>
<col> username </col> <col> Text </col> <col> <para/> </col>
<col>
- The username of the user that is involved in the action.
+ The username of the user that is involved in the action. This field refers to the table <strong>usr</strong>
</col>
</row>
<row>
<col> notificationid </col> <col> Bigint </col> <col> 8 </col>
<col>
- Reference to the notification.
+ Reference to the notification. This will link the action to the notification.
</col>
</row>
<row>
<col> timestamp </col> <col> Timestamp </col> <col> <para/> </col>
<col>
- The time when the action has been entered into the system. This is the time without the timezone<footnote>The timestamp without time has been selected, since this is the system time. To have the system functioning without any physical borders one of the settings on the system is time in GMT (UTC). This ensures also the added value of the system log.</footnote>. This will be automatically added when the record is added into the database.
+ The time when the action has been entered into the system.
+ This is the time without the timezone<footnote>The timestamp without
+ time has been selected, since this is the system time.
+ To have the system functioning without any physical borders one
+ of the settings on the system is time in GMT (UTC).
+ This ensures also the added value of the system log.</footnote>.
+ This will be automatically added when the record is added into the database.
</col>
</row>
<row>
<col> statuscode </col> <col> Varchar </col> <col> 3 </col>
<col>
- The status of the Notification
+ The status of the Notification AFTER this action has been taken.
</col>
</row>
<row>
<col> remarks </col> <col> Text </col> <col> <para/> </col>
<col>
- Remarks entered by the user if it concerns a manual action or the text of automatically generated warnings.
+ Remarks entered by the user if it concerns a manual action
+ or the text of automatically generated warnings.
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<col> anu_statuscode </col> <col> statuscode </col> <col> <para/> </col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
+<para>
+Relationships with other tables:
+</para>
<table cpos='llp{7cm}'>
<thead>
<col> Fieldname </col> <col> Remote Table </col>
<row>
<col> notificationid </col> <col> notification </col>
<col>
- Each action that takes place is registered in this table. By using the notification the relevant notification. The relationship has to be enforced.
+ Each action that takes place is registered in this table.
+ By using the notification the relevant notification. The relationship has to be enforced.
</col>
</row>
<row>
<col> username </col> <col> user </col>
<col>
- Each step in the process has to be related to the user. If the system itself generates an action the user will be <strong>gnucomo</strong><footnote>This implies that the system automatically has an username gnucomo.</footnote>.
+ Each step in the process has to be related to the user.
+ If the system itself generates an action the user will be
+ <strong>gnucomo</strong><footnote>This implies that the system
+ automatically has an username gnucomo.</footnote>.
</col>
</row>
</table>
<picture src="erd-anu.png" eps="erd-anu"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Sample data.</heading>
<para>
Since no data is delivered automatically a couple of sample records are shown here.
</para>
- <table cpos='llllllp{4cm}'>
+ <table cpos='p{1cm}p{1cm}p{1cm}lllp{4cm}'>
<thead>
<col> Actionstepid </col> <col> Actionid </col> <col> Notificationid </col>
<col> Username </col> <col> Timestamp </col> <col> Status </col>
<col> 3 </col> <col> 5 </col> <col> 1 </col> <col> Gnucomo </col>
<col> 2002-07-14 16:14:09 </col> <col> OPN </col>
<col>
- <para>Automatic e-mail to user: <reference href="mailto:brenno@dewinter.com">brenno@dewinter.com</reference>:</para>
- <para>Gnucomo detected a portscan on system <reference href="http://gnucomo.dewinter.com/">gnucomo.dewinter.com</reference></para>
+ <para>Automatic e-mail to user:
+ <reference href="mailto:brenno@dewinter.com">brenno@dewinter.com</reference>:</para>
+ <para>Gnucomo detected a portscan on system
+ <reference href="http://gnucomo.dewinter.com/">gnucomo.dewinter.com</reference></para>
</col>
</row>
<row>
<col> 8 </col> <col> 3 </col> <col> 1 </col> <col> Brenno </col>
<col> 2002-07-14 16:24:59 </col> <col> PEN </col>
<col>
- Services tables learns me that all services are aimed at Windows-based services. Attempts for platform specific expoits.
+ Services tables learns me that all services are aimed at Windows-based services.
+ Attempts for platform specific expoits.
</col>
</row>
<row>
</col>
</row>
</table>
- </paragraph>
-</subsubsection>
+ </subsubsection>
+</subsection>
+
+<subsection>
+<heading>history</heading>
+<para>
+The history table records all changes to properties of parameters.
+</para>
<subsubsection>
- <heading>log & log_adv. </heading>
+<heading>The fields</heading>
+<para>
+The fields of the <emph>history</emph> table are listed below:
+</para>
+<table cpos='lllp{6cm}'>
+ <thead>
+ <col> Fieldname </col> <col> Fieldtype </col> <col> Size </col>
+ <col> Remarks </col>
+ </thead>
+ <row>
+ <col>paramid</col><col>bigint</col><col>8</col>
+ <col>The parameter to which this history belongs. Refers to the parameter table</col>
+ </row>
+ <row>
+ <col>modified</col><col>timestamp</col><col> </col>
+ <col>Time at which the property value or parameter changed</col>
+ </row>
+ <row>
+ <col>change_nature</col><col>enum</col><col> </col>
+ <col>Parameter created to destroyed; property value changed</col>
+ </row>
+ <row>
+ <col>changed_property</col><col>text</col><col> </col>
+ <col>Name of the parameter's property that changed.</col>
+ </row>
+ <row>
+ <col>new_value</col><col>text</col><col> </col>
+ <col>The new actual value of the property at the time of modification</col>
+ </row>
+ <row>
+ <col>remark</col><col>text</col><col> </col>
+ <col>A short explanation of why the property changed</col>
+ </row>
+</table>
<para>
-To store the log-data there are two tables in use that have a one-on-one relationship. The logic behind this is the difference between raw-always needed data and the somewhat processed data to support basic retrieval. The last category isn't always used and when it's used it is redundant. Also the raw log is very important to the integrity of the system. For these reasons the processed data has been physically separated in a second table called <emph>log_adv</emph>. If needed a view will be available that combines the two tables. Despite the load indexing on <emph>log_adv</emph> will be done thoroughly.
+Each time something about a parameter changes, this is recorded in the
+change history of the paraneter.
+When such a change happens, one of three things may occur to a parameter,
+as stated in the 'change_nature' field:
+
+<enumerate>
+<item>A new parameter is created.</item>
+<item>The value of one of the properties was altered.</item>
+<item>The parameter is removed.</item>
+</enumerate>
+
+When a parameter is created or destroyed, the fields 'changed_property' and
+'new_value' are irrelevant.
</para>
+
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
+ <heading>log & log_adv. </heading>
+
+<para>
+To store the log-data there are two tables in use that have a one-on-one relationship.
+The logic behind this is the difference between raw-always needed data and the somewhat
+processed data to support basic retrieval.
+The last category isn't always used and when it's used it is redundant.
+Also the raw log is very important to the integrity of the system.
+For these reasons the processed data has been physically separated in a
+second table called <emph>log_adv</emph>.
+If needed a view will be available that combines the two tables.
+Despite the load indexing on <emph>log_adv</emph> will be done thoroughly.
+</para>
+</subsection>
+
+<subsection>
<heading>log.</heading>
-<paragraph>
+<subsubsection>
<heading>The fields of log.</heading>
<para>
-The fields in log are focussed around the raw data and the data needed to link this to other tables in the system.
+The fields in log are focussed around the raw data arriving from log-files and output
+from processes that deliver status information.
+The log table is the one and only place used to automatically deliver data to the gnucomo-system.
+This makes the facts very traceable and ensures that there is one point where the data isn't
+yet fragmented (usefull if new insights are usefull). The log table also provides the
+integrity of the data as it was delivered. The data will be needed to provide
+a link to other tables in the system that handle processed derrivates of the
+original data (added intelligence).
</para>
<table cpos='lllp{6cm}'>
<row>
<col> original_filename </col> <col> Text </col> <col> <para/> </col>
<col>
- This field refers to the filename that contained this entry. The original entries as received by the gnucomo-server (flat files). The files are sent in batches, which makes it very hard to find where the original logline is. This will enable to see the files to detect bugs in gnucomo if any occur. It may well be that in a later stage this functionality becomes obsolete.
+ This field refers to the filename that contained this entry.
+ The original entries as received by the gnucomo-server (flat files).
+ The files are sent in batches, which makes it very hard to find where the
+ original logline is. This will enable to see the files to detect
+ bugs in gnucomo if any occur.
+ It may well be that in a later stage this functionality becomes obsolete.
</col>
</row>
<row>
<col> servicecode </col> <col> Text </col> <col> <para/> </col>
<col>
This field explains what service was recognized (for instance 'kernel, 'httpd' or 'smtp')
+ this make later processing easier.
</col>
</row>
<row>
<col> type_of_logid </col> <col> Bigint </col> <col> 8 </col>
<col>
- Reference to the table type_of_log that contains information on what type of log/report we have here (how gnucomo recognized it).
+ Reference to the table type_of_log that contains information on what
+ type of log/report we have here (how gnucomo recognized it).
</col>
</row>
<row>
<col> object_timestamp </col> <col> Timestamp </col> <col> 8 </col>
<col>
- Timestamp as has been written into the log.
+ Timestamp of the moment the data was written into the log-file itself.
+ This is the time on the remote system.
</col>
</row>
<row>
<col> timestamp </col> <col> Timestamp </col> <col> <para/> </col>
<col>
- The time when the action has been entered into the system. This is the time without the timezone<footnote>The timestamp without time has been selected, since this is the system time. To have the system functioning without any physical borders one of the settings on the system is time in GMT (UTC). This ensures also the added value of the system log.</footnote>. This will be generated upon entry into the database.
+ The time when the action has been entered into the system.
+ This is the time without the timezone<footnote>The timestamp without time
+ has been selected, since this is the system time.
+ To have the system functioning without any physical borders one of
+ the settings on the system is time in GMT (UTC).
+ This ensures also the added value of the system log.</footnote>.
+ This will be generated upon entry into the database.
</col>
</row>
<row>
<col> rawdata </col> <col> TEXT </col> <col> <para/> </col>
<col>
- The raw log data
+ The raw log data as it was handed to gnucomo.
</col>
</row>
+ <row>
+ <col>processed</col><col>BOOLEAN </col> <col> <para/> </col>
+ <col>This record is a flag (true or false).
+ If a record that has been entered into the log-table has been processed
+ (mostly into several log_adv-tables) this flag is set to true.
+ The flag indicates that gcm_daemon processed the record.
+ This doesn't nescessarily mean that they have been used.
+ By default the setting is false.
+ So the flag helps in the detection of new arrivals in the log-table.</col>
+ </row>
+ <row>
+ <col>recognized</col><col>BOOLEAN</col> <col><para/> </col>
+ <col>This record is a flag (true or false).
+ If the processing by gcm_daemon of this particular record has been done the flag
+ will only be set to true when gcm_daemon could process the record.
+ If there was no support for this particular entry the status remains false.
+ The main aim of this functionality is relevant when a new version of the daemon is present.
+ Unrecognized records will then be transferred to unprocessed again and a
+ new attempt will be made to process these records.</col>
+ </row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<col> log_timestamp </col> <col> timestamp </col> <col> <para/> </col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<row>
<col> type_of_logid </col> <col> type_of_log </col>
<col>
- Each logbook has a certain type of reporting. This explains what type of log was received (and thus which rules for detection was applied).
+ Each logbook has a certain type of reporting.
+ This explains what type of log was received (and thus which rules for detection was applied).
</col>
</row>
<row>
<col> systemuser </col> <col> user </col>
<col>
- Links a registered user of an object to this log entry<footnote>Upon entry of an object most of the times the passwd-file (UNIX-systems) or the userlist will serve as the entrypoint of users. Non existing users will be added and will have to be verified by an administrator before this entry become definite.</footnote>.
+ Links a registered user of an object to this log
+ entry<footnote>Upon entry of an object most of the times the
+ passwd-file (UNIX-systems) or the userlist will serve as the entrypoint of users.
+ Non existing users will be added and will have to be verified by an
+ administrator before this entry become definite.</footnote>.
</col>
</row>
</table>
</para>
-</paragraph>
-<paragraph>
+</subsubsection>
+<subsubsection>
+<heading>Log_adv</heading>
+<para>
+As soon as new data is detected in the log-table processing takes place.
+If the data is recognized by the gcm_daemon one or more log_adv records
+will be written where data is separated. Due to the high diversity multiple
+types of the log_adv table will be created.
+All of them will be an inheritance of the log_adv-table.
+</para>
+</subsubsection>
+<subsubsection>
<heading>The fields of log_adv.</heading>
<para>
</col>
</thead>
<row>
- <col> logid </col> <col> Bigint </col> <col> 8 </col>
- <col>
- Bigint (eight bit) 1-to-1 relationship with log
- </col>
- </row>
- <row>
- <col> source_ip </col> <col> Inet </col> <col> <para/> </col>
- <col>
- This is the IP-address (V4) of the host that sended the data.
+ <col>log_advid</col><col>Bigint autonumber</col><col>8</col>
+ <col>Since some records can generate several log_adv records a unique identifyer is needed.
+ This record provides that.
</col>
</row>
<row>
- <col> destination_ip </col> <col> Inet </col> <col> <para/> </col>
- <col>
- This is the IP-address (V4) of the host that was targetted.
- </col>
- </row>
- <row>
- <col> mac_address </col> <col> Macaddr </col> <col> <para/> </col>
- <col>
- This is the MAC-address logged
- </col>
- </row>
- <row>
- <col> packetlength </col> <col> Int </col> <col> <para/> </col>
- <col>
- The length of the packet
- </col>
- </row>
- <row>
- <col> protocol </col> <col> Text </col> <col> <para/> </col>
- <col>
- The protocol used for transmission(mostly TCP/UDP/ICMP).
- </col>
- </row>
- <row>
- <col> source_port </col> <col> Int </col> <col> <para/> </col>
- <col>
- The portnumber used at origin.
- </col>
- </row>
- <row>
- <col> destination_port </col> <col> Int </col> <col> <para/> </col>
- <col>
- The portnumber for the target
- </col>
- </row>
- <row>
- <col> messageid </col> <col> Text </col> <col> <para/> </col>
- <col>
- Messageid for e-mails
- </col>
- </row>
- <row>
- <col> system_username </col> <col> Text </col> <col> <para/> </col>
- <col>
- Username on the object
- </col>
- </row>
- <row>
- <col> networkdevice </col> <col> Text </col> <col> <para/> </col>
+ <col> logid </col> <col> Bigint </col> <col> 8 </col>
<col>
- When this is about network-traffic the device that worked the data
+ Bigint (eight bit) 1-to-1 relationship with log
</col>
</row>
</table>
-</paragraph>
-<paragraph>
+</subsubsection>
+
+<subsubsection>
<heading>The indexes.</heading>
<para>
<col> networkdevice </col> <col> networkdevice </col> <col> <para/> </col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Sample data combined from log and log_adv.</heading>
<para>
-The sample data derrived here has been gathered in logs. Since the tablestructure is very long the representation is somewhat different:
+The sample data derrived here has been gathered in logs.
+Since the tablestructure is very long the representation is somewhat different:
</para>
<table cpos='lp{8cm}'>
<col> Networkdevice </col> <col> eth0 </col>
</row>
</table>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>log_notification. </heading>
<para>
-In the log_notification the logbook entries that have caused an alert to occur are saved. When this table is used something has been detected. As this is clearly an intermediate table we anticipate to design checks where multiple entries in a log-file can lead to one notification. For forensics indexing will be focussed on retrieval speed.
+In the log_notification the logbook entries that have caused an alert to occur are saved.
+When this table is used something has been detected.
+As this is clearly an intermediate table we anticipate to design checks where
+multiple entries in a log-file can lead to one notification.
+For forensics indexing will be focussed on retrieval speed.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</row>
</table>
-</paragraph>
-<paragraph>
+</subsubsection>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<col> lon_logid </col> <col>logid </col> <col> <para/> </col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<para>
<picture src="erd-lognotif.png" eps="erd-lognotif"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Sample data.</heading>
<para>
Some sample data:
<col> 11 </col> <col> 2 </col>
</row>
</table>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>notification. </heading>
<para>
-In this table all detected issues per object will be written. Issues are entered based on immediate detection, periodical detection or manually. Since this table is mostly used to work from in the interface being in control is crucial. When systems function properly more retrieval than data entry will take place. Also data retrieval will be done in all sorts of ways. Indexing must be huge to facilitate that.
+The main task of gnucomo is detection issues.
+As soon as something has been detected based on the queries run a notification will be created.
+This notification will be the warning towards the system that something has been detected.
+Issues are entered based on immediate detection, periodical detection or manually.
+In an ideal case no notification will occur, but as time goes by issues will occur.
+When systems function properly more retrieval than data entry will take place.
+Also data retrieval will be done in all sorts of ways. One can think of immediate display,
+but also updates due to actions and ultimately reporting.
+Indexing must be huge to facilitate all these queries.
+
+There will be one single point-of-contact with the gnucomo-system.
+This point-of-contact will be the XML/SOAP interface.
+By doing that we can set user-rights (as has been impleted in this table)
+without communicating that to the user.
+Also the gnucomo-XML/SOAP-interface will verify if the attempted action is allowed.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</thead>
<row>
- <col> notificationid </col> <col> Bigserial </col> <col> 8 </col>
+ <col> notificationid </col> <col> Bigint Autonumber </col> <col> 8 </col>
<col>
Autonumber
</col>
</col>
</row>
<row>
- <col> type_of_notification_id </col> <col> Bigint </col> <col> 8 </col>
+ <col> type_of_issue_id </col> <col> Bigint </col> <col> 8 </col>
<col>
- Reference to the <emph>type_of_notification</emph> indicating what type of notification we have here and what basic rules apply.
+ Reference to the <emph>type_of_issue</emph> indicating what
+ type of notification we have here and what basic rules apply.
+ Notifications are always created on an issue that can be raised.
</col>
</row>
<row>
<col> timestamp </col> <col> Timestamp </col> <col> <para/> </col>
<col>
- Timestamp that this notification was created.
+ Timestamp used to indicate the moment when this notification was created.
</col>
</row>
<row>
<col> statuscode </col> <col> Varchar </col> <col> 3 </col>
<col>
The status the actual status a notification has.
+ This can be new, open, pending, waiting for verification, rejected, closed, needs investigation
</col>
</row>
<row>
<col> priority </col> <col> Int </col> <col> 4 </col>
<col>
- The priority that is given to this issue<footnote>Basically there will be five priority levels. The rule is that the lower the number gets the more urgent the issue is. At the moment the priority level is set by the system pre-defined actions will take place.</footnote>.
+ The priority that is given to this issue<footnote>Basically there will be
+ five priority levels. The rule is that the lower the number gets the more
+ urgent the issue is. At the moment the priority level is set by the system
+ pre-defined actions will take place.</footnote>.
</col>
</row>
<row>
<col> escalation_count_timestamp </col> <col> Timestamp </col> <col> <para/> </col>
<col>
- Timestamp since the last escalation took place<footnote>The system offers the possibility to automatically escalate issue if no action has been taken within a certain amount of time. Based on the status and the type of notification escalation can take place.</footnote>.
+ Each notification has a basic priority-level but if certain time-constraints
+ are not met it is possible to do an automatic escalation.
+ By doing that new rules may apply and more priority can be given.
+ This field has the timestamp since the last escalation took
+ place<footnote>The system offers
+ the possibility to automatically escalate issue if no action has been
+ taken within a certain amount of time. Based on the status and the type
+ of notification escalation can take place.</footnote>.
</col>
</row>
<row>
<col> repeat_notification_timestamp </col> <col> Timestamp </col> <col> <para/> </col>
<col>
- Timestamp at which moment in time a repeat notification should occur<footnote>After this time has passed a notification is being resent. After a resent automatically a new time is set for another resent. If any actions takes place (except automated entries of course) the time is emptied so that an administrator will no longer be bothered by the system.</footnote>.
+ Timestamp at which moment in time a repeat notification should occur<footnote>After
+ this time has passed a notification is being resent.
+ After a resent automatically a new time is set for another resent.
+ If any actions takes place (except automated entries of course)
+ the time is emptied so that an administrator will no longer be bothered by the system.</footnote>.
</col>
</row>
<row>
<col> securitylevel_view </col> <col> Int </col> <col> 4 </col>
<col>
- The securitylevel that is needed to view this entry.
+ The securitylevel that is needed to be allowed to view this entry. This field builds the opportunity to have certain specific issues only handled by certain persons. <footnote>This feature may be beneficial to make monitoring more discrete. For instance if a check is being done on who is looking on pornographic websites privacy issues play as well. By setting this value only certain users can undertake action. Something else would be a feature to have fraud protection. If this happens it goes beyond the work of a normal administrator and only certain officers are allowed to deal with these issues. One last example would also be privacy related. Suppose a system logs all the unix-commands given by a certain user. If something is found based on the actions of the users a protection mechanism is needed. Only special users are allowed to view this notification.</footnote>
</col>
</row>
<row>
<col> securitylevel_add </col> <col> Int </col> <col> 4 </col>
<col>
- Securitylevel to add information to this item or to undertake action
+ There is a fundamental difference between letting users see notification and actually work on notifications. This field indicates what security level needs to be present to edit this notification.
</col>
</row>
<row>
<col> securitylevel_close </col> <col> Int </col> <col> 4 </col>
<col>
- The securitylevel needed to be able to close this notification.
+ The securitylevel needed to be able to close this notification. After working on an issue one ultimately hope to close the notification. This field indicates who is allowed to do so. If one attempts to close the issue, but is lacking sufficient rights the status will be changed to 'waiting for verification'. This will enable the superuser to quickly tick of the list of issues to be verified and still be in control.
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<col> repeat_notification_timestamp </col> <col> <para/> </col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<para>
<picture src="erd-notif.png" eps="erd-notif"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Sample data.</heading>
<para>
<col> Securitylevel_close </col> <col> 4 </col>
</row>
</table>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>object</heading>
<para>
-The <emph>object</emph> table contains general information on the objects being monitored. The table object will more be used for retrieval, since adding objects will be an occasional process. Anything that for some reason can be indexed ought to be indexed.
+The <emph>object</emph> table contains general information on the objects being monitored. Mostly objects will be computers, but it may also be something else in the future like routers, switches, gateways, PDA's or other devices. The table object will more be used for retrieval, since adding objects will be an
+occasional process. Anything that for some reason can be indexed ought to be indexed.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</thead>
<row>
- <col> objectid </col> <col> Bigserial </col> <col> 8 </col>
+ <col> objectid </col> <col> Bigint Autonumber </col> <col> 8 </col>
<col>
Autonumbering code for the computer or device.
</col>
<row>
<col> objectname </col> <col> Text </col> <col> <para/> </col>
<col>
- The hostname of the object
+ The hostname of the object as it can be recognized by the gcm_input-application.
</col>
</row>
<row>
<col> objectcode </col> <col> Text </col> <col> <para/> </col>
<col>
- Unique identifier (if existent) on the system<footnote>In Linux this will typically be the <emph>hostid</emph>.</footnote>.
+ Unique identifier (if existent) on the system<footnote>In Linux this will
+ typically be the <emph>hostid</emph>.</footnote>.
</col>
</row>
<row>
<para/>
</col>
<col>
- Description of the object. What type of system is it
+ Description of the object. What type of system is it, what specifics are there to know, etc.
</col>
</row>
<row>
<row>
<col> timezone </col> <col> Text </col> <col> <para/> </col>
<col>
- The timezone where this object is located<footnote>For objects that move around like PDA's and laptops the timezone can be the place where a person is stationed or better GMT.</footnote>.
+ The timezone where this object is located<footnote>For objects that
+ move around like PDA's and laptops the timezone can be the place where
+ a person is stationed or better GMT.</footnote>.
</col>
</row>
<row>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<col> obj_mail_from </col> <col> mail_from </col> <col> <para/> </col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<row>
<col> <para/> </col> <col> Notification </col>
<col>
- Reference to the table <emph>notification</emph> that contains the notifications that have been created.
+ Reference to the table <emph>notification</emph> that contains the notifications
+ that have been created.
</col>
</row>
<row>
<row>
<col> <para/> </col> <col> Object_priority </col>
<col>
- Reference to the <emph>object_priority</emph> table that indicates how a certain level of priority has to be dealt with.
+ Reference to the <emph>object_priority</emph> table that indicates how a
+ certain level of priority has to be dealt with.
</col>
</row>
<row>
<para>
<picture src="erd-object.png" eps="erd-object"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Sample data.</heading>
<para>
-There is no preset data and therefor it's an example has been created. The table has quite some fields so the example has the fieldname in the left column and the data in the right column.
+There is no preset data and therefor it's an example has been created.
+The table has quite some fields so the example has the fieldname in the
+left column and the data in the right column.
</para>
-<table cpos='ll'>
+<table cpos='lp{6cm}'>
<thead>
- <col>
- <para>Fieldname</para>
- </col>
- <col>
- <para>Sample data</para>
- </col>
+ <col> Fieldname </col>
+ <col> Sample data </col>
</thead>
<row>
- <col>
- <para>Objectid</para>
- </col>
- <col>
- <para>1</para>
- </col>
+ <col> Objectid </col>
+ <col> 1 </col>
</row>
<row>
- <col>
- <para>Objectname</para>
- </col>
- <col>
- <para>webber.dewinter.com</para>
- </col>
+ <col> Objectname </col>
+ <col> webber.dewinter.com </col>
</row>
<row>
- <col>
- <para>Objectcode</para>
- </col>
- <col>
- <para>7f0100</para>
- </col>
+ <col> Objectcode </col>
+ <col> 7f0100 </col>
</row>
<row>
- <col>
- <para>Scp_enabled</para>
- </col>
- <col>
- <para>T</para>
- </col>
+ <col> Scp_enabled </col>
+ <col> T </col>
</row>
<row>
- <col>
- <para>Scp_inet</para>
- </col>
- <col>
- <para>192.168.221.212</para>
- </col>
+ <col> Scp_inet </col>
+ <col> 192.168.221.212 </col>
</row>
<row>
- <col>
- <para>Mail_enabled</para>
- </col>
- <col>
- <para>T</para>
- </col>
+ <col> Mail_enabled </col>
+ <col> T </col>
</row>
<row>
- <col>
- <para>Mail_from</para>
- </col>
- <col>
- <para>
- <reference href="mailto:gnucomo@maintenance.dewinter.com">gnucomo@maintenance.dewinter.com</reference>
- </para>
- </col>
+ <col> Mail_from </col>
+ <col> <reference href="mailto:gnucomo@maintenance.dewinter.com">gnucomo@maintenance.dewinter.com</reference> </col>
</row>
<row>
- <col>
- <para>Sms_enabled</para>
- </col>
- <col>
- <para>T</para>
- </col>
+ <col> Sms_enabled </col>
+ <col> T </col>
</row>
<row>
- <col>
- <para>Sms_number</para>
- </col>
- <col>
- <para>06-XXXXXXXX</para>
- </col>
+ <col> Sms_number </col>
+ <col> 06-XXXXXXXX </col>
</row>
<row>
- <col>
- <para>Fax_enabled</para>
- </col>
- <col>
- <para>T</para>
- </col>
+ <col> Fax_enabled </col>
+ <col> T </col>
</row>
<row>
- <col>
- <para>Fax_number</para>
- </col>
- <col>
- <para>0318-XXXXXX</para>
- </col>
+ <col> Fax_number </col>
+ <col> 0318-XXXXXX </col>
</row>
<row>
- <col>
- <para>Object_description</para>
- </col>
- <col>
- <para>19 inch 4 units, AMD-300 with two 27Gb disks (RAID-0), 256Mb memory</para>
- </col>
+ <col> Object_description </col>
+ <col> 19 inch 4 units, AMD-300 with two 27Gb disks (RAID-0), 256Mb memory </col>
</row>
<row>
- <col>
- <para>Object_owner</para>
- </col>
+ <col> Object_owner </col>
<col>
<para>Brenno de Winter</para>
<para>De Winter Information Solutions</para>
</col>
</row>
<row>
- <col>
- <para>Physical_location</para>
- </col>
+ <col> Physical_location </col>
<col>
<para>Internet Provider XYZ</para>
<para>Your street here 38</para>
<para>9999 XX YOUR CITY</para>
<para>THE NETHERLANDS</para>
<para>Phone: +31 XXX XXX XXX</para>
- <para/>
+
<para>Dataroom. System: Q7845</para>
</col>
</row>
<row>
+ <col> Remark </col>
<col>
- <para>Remark</para>
- </col>
- <col>
- <para>A replacement system is available at the office location. The following persons have been authorized to enter the data room at the ISP:</para>
+ <para>A replacement system is available at the office location.
+ The following persons have been authorized to enter the data room at the ISP:</para>
<para>* Arjen Baart</para>
<para>* Peter Busser</para>
<para>* Brenno de Winter</para>
</row>
</table>
<para/>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>object_issue. </heading>
<para>
-This table will store the policy on a certain issue like the priority being recognized and special actions to take. Since policies are utilized by the systems continuously all other process will rely on this index, while users will change the values occasionally.
+This table will store the policy on a certain issue like the priority being
+recognized and special actions to take. Since values in this table are bound to the object, responses to the same incident can lead to different alert levels based on the importance of the object.
+Since policies are utilized by the systems continuously all other process will rely on this index,
+while users will change the values occasionally.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</row>
<row>
<col>
- <para>type_of_notificationid</para>
+ <para>type_of_issueid</para>
</col>
<col>
<para>Bigint</para>
<para>8</para>
</col>
<col>
- <para>Reference to the <emph>type_of_notification</emph> indicating what type of notification we have here and what basic rules apply.</para>
+ <para>Reference to the <emph>type_of_issue</emph> indicating
+ how we will handle this type of issue for this particular object. If nothing is entered here, the default rules apply.</para>
</col>
</row>
<row>
<para>4</para>
</col>
<col>
- <para>The priority that will be set automatically when this type of notification is entered into the system.</para>
+ <para>The priority that will be set automatically when this type of
+ issue is entered into the system.</para>
</col>
</row>
<row>
<para/>
</col>
<col>
- <para>The maximum priority given to this type of notification.</para>
+ <para>The maximum priority given to this type of notification. This will prevent an endless escalation of the issue. No priority can be higher than 1.</para>
</col>
</row>
<row>
<para/>
</col>
<col>
- <para>Some checks can have a special settings (for instance alert after 5 failed login attempts instead of 3).</para>
+ <para>Some checks can have a special settings (for instance alert
+ after 5 failed login attempts instead of 3).</para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<para/>
</col>
<col>
- <para>type_of_notification_id</para>
+ <para>type_of_issue_id</para>
</col>
<col>
<para>Primary key</para>
</row>
<row>
<col>
- <para>obi_type_of_notificationid</para>
+ <para>obi_type_of_issueid</para>
</col>
<col>
- <para>type_of_notification_id</para>
+ <para>type_of_issue_id</para>
</col>
<col>
<para/>
</col>
</row>
</table>
- </paragraph>
+ </subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<picture src="erd-objissue.png" eps="erd-objissue"/>
</para>
-</paragraph>
-<paragraph>
+</subsubsection>
+<subsubsection>
<heading>Sample data.</heading>
<para>
<para>Objectid</para>
</col>
<col>
- <para>Type_of_notification_id</para>
+ <para>Type_of_issue_id</para>
</col>
<col>
<para>Default Priority</para>
</col>
</row>
</table>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
- <heading>object_priority. </heading>
+<subsection>
+ <heading>object_priority.</heading>
<para>
-This table stores per object how a certain level of priority is being dealt with. What policies do apply. This table is mostly used for retrieval, so firm indexing is logic.
+By default a prioritycode has a certain meaning. But default behaviour can be used for most cases, but not all. For some objects a deviation would be usefull. For instance a very important webserver under an attack should maybe alarm more agressive than a printer-server that is used only marginally. This table stores per object how a certain level of priority is being dealt with. The main issue is the question: What policies do apply? If nothing is available default behaviour as defined in the table priority will apply. This table is mostly used for retrieval, so firm indexing is logic.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
<para/>
</col>
<col>
- <para>Repeat this notification if no action occurs since the notification. Yes = T / No = F</para>
+ <para>Repeat this notification if no action occurs within a certain timeframe. Yes = T / No = F</para>
</col>
</row>
<row>
</col>
</row>
</table>
- </paragraph>
+ </subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
</row>
<row>
<col>
- <para>obi_type_of_notification_id</para>
+ <para>obi_type_of_issue_id</para>
</col>
<col>
- <para>type_of_notification_id</para>
+ <para>type_of_issue_id</para>
</col>
<col>
<para/>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<para>
<picture src="erd-objprior.png" eps="erd-objprior"/>
</para>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>object_service</heading>
<para>
-The object service table indicates which services can be expected on the system If input fails to show up a notification can be generated.
+The object service table indicates which services should report itself to the Gnucomo-system.
+If input fails to show up a notification can be generated.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
<para>8</para>
</col>
<col>
- <para>The expected interval in minutes between two log entries. If this gives a time-out a notification is generated<footnote>To avoid many false positives it may be wise to give the system always 1 or 2 minutes extra time. If for some reason a connection is slow or a mail-daemon restarted the effect would generate tuns of notifications.</footnote>. The following values can be considered the most common:</para>
+ <para>The expected interval in minutes between two log entries.
+ If this gives a time-out a notification is generated<footnote>To avoid many
+ false positives it may be wise to give the system always 1 or 2 minutes extra time.
+ If for some reason a connection is slow or a mail-daemon restarted
+ the effect would generate tuns of notifications.</footnote>.
+ The following values can be considered the most common:</para>
<para>* 60 hourly entries</para>
<para>* 120 two hourly entries</para>
<para>* 240 four hourly entries</para>
<para/>
</col>
<col>
- <para>The timestamp of the last entry (for detecting exceeded interval). This field could be derived from the log-table as well, but the redundance gives a performance on detection that is useful, since a check should run every minute.</para>
+ <para>The timestamp of the last entry (for detecting exceeded interval).
+ This field could be derived from the log-table as well, but the
+ redundance gives a performance on detection that is useful, since a
+ check should run every minute.</para>
</col>
</row>
<row>
<para/>
</col>
<col>
- <para>If a service hasn't been set, the application user should indicate that this is valid (logs shouldn't just appear). New entries will be added automatically but still have to be verified.</para>
+ <para>If a service hasn't been set, the application user should
+ indicate that this is valid (logs shouldn't just appear).
+ New entries will be added automatically but still have to be verified.</para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<para>
<picture src="erd-objservice.png" eps="erd-objservice"/>
</para>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>object_system_user</heading>
<para>
-This table will derive a list of users that can be identified based on the log-files in the system. This table is filled during data entry. But the filling of the table is dependent on the fact if the user has been entered before. So during the processing the read will be done more than the data entry and that makes heavy indexing logic.
+Every object knows users either by the security of the object itself or through a central database (like LDAP for instance). Initially a new object can sent the userlist to gnucomo. Any username that isn't in the userlist is potentially dangerous. To avoid any mis-understandings this user is not a gnucomo-user, but a user on a remote system. So during the processing the read will be done more than the data entry (one account will most likely do multiple actions) and
+that makes heavy indexing logic.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</row>
</table>
- </paragraph>
+ </subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
<para>osu_pk</para>
</col>
<col>
- <para>objectid</para>
+ <para>objectid, system_username</para>
</col>
<col>
<para>Primary key</para>
</col>
</row>
</table>
- </paragraph>
+ </subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<para>
<picture src="erd-objsysusr.png" eps="erd-objsysusr"/>
</para>
- </paragraph>
-</subsubsection>
+ </subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>object_user</heading>
<para>
-This table will enable users to get access to the information belonging to an object. Also this table is mainly used for data retrieval and will rely on the indexes.
+This table will enable users to get access to the information belonging to an object.
+Also this table is mainly used for data retrieval and will rely on the indexes.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
The indices of the <emph>object_user</emph> table:
</col>
</row>
<row>
- <col>
- <para/>
- </col>
- <col>
- <para>username</para>
- </col>
- <col>
- <para>Primary key</para>
- </col>
+ <col>
+ <para/>
+ </col>
+ <col>
+ <para>username</para>
+ </col>
+ <col>
+ <para>Primary key</para>
+ </col>
+ </row>
+ <row>
+ <col>
+ <para>ous_objectid</para>
+ </col>
+ <col>
+ <para>objectid</para>
+ </col>
+ <col>
+ <para/>
+ </col>
+ </row>
+ <row>
+ <col>
+ <para>ous_username</para>
+ </col>
+ <col>
+ <para>username</para>
+ </col>
+ <col>
+ <para/>
+ </col>
+ </row>
+ <row>
+ <col>
+ <para>ous_security_level</para>
+ </col>
+ <col>
+ <para>ous_security_level</para>
+ </col>
+ <col>
+ <para/>
+ </col>
+ </row>
+ </table>
+</subsubsection>
+
+<subsubsection>
+ <heading>The relationships.</heading>
+
+<para>
+Relationships with other tables:
+</para>
+
+<table cpos='llp{8cm}'>
+ <thead>
+ <col>
+ <para>Fieldname</para>
+ </col>
+ <col>
+ <para>Remote Table</para>
+ </col>
+ <col>
+ <para>Remarks</para>
+ </col>
+ </thead>
+ <row>
+ <col>
+ <para>objectID</para>
+ </col>
+ <col>
+ <para>object</para>
+ </col>
+ <col>
+ <para>Reference to the object (which object does this apply to).</para>
+ </col>
+ </row>
+ <row>
+ <col>
+ <para>username</para>
+ </col>
+ <col>
+ <para>user</para>
+ </col>
+ <col>
+ <para>Reference to the user.</para>
+ </col>
+ </row>
+ </table>
+
+ <para>In the model this look like this:</para>
+ <picture src="erd-objusr.png" eps="erd-objusr"/>
+
+</subsubsection>
+</subsection>
+
+<subsection>
+<heading>parameter</heading>
+<para>
+The parameter table stores the operational parameters of a monitored object.
+The parameters of an object describe the object's resources and configurations.
+For each object, a large set of parameters can be defined. They range from
+anything like file systems and installed packages to the system's users.
+</para>
+
+<subsubsection>
+<heading>The fields</heading>
+<para>
+The fields of the <emph>parameter</emph> table are listed below:
+</para>
+<table cpos='lllp{6cm}'>
+ <thead>
+ <col> Fieldname </col> <col> Fieldtype </col> <col> Size </col>
+ <col> Remarks </col>
+ </thead>
+ <row>
+ <col>paramid</col><col>bigserial</col><col>8</col>
+ <col>Uniquely identifies the parameter. Used in property and history tables.</col>
+ </row>
+ <row>
+ <col>objectid</col><col>bigint</col><col>8</col>
+ <col>The object of which this is a parameter. Refers to the object table.</col>
+ </row>
+ <row>
+ <col>name</col><col>text</col><col> </col>
+ <col>Name of the parameter to identify the resource</col>
+ </row>
+ <row>
+ <col>class</col><col>text</col><col> </col>
+ <col>Similar parameters are in the same class. Refers to the
+ <emph>parameter_class</emph> table.
+ </col>
+ </row>
+ <row>
+ <col>description</col><col>text</col><col> </col>
+ <col>A verbose description of the parameter</col>
+ </row>
+
+</table>
+
+<para>
+The combination of objectid, name and class must be unique.
+</para>
+
+</subsubsection>
+<subsubsection>
+<heading>Sample data</heading>
+
+<para>
+The table below lists a few examples of parameters
+</para>
+<table cpos='lllll'>
+ <thead>
+ <col>paramid</col><col>objectid</col><col>name</col><col>class</col><col>description</col>
+ </thead>
+ <row>
+ <col>1</col><col>1</col><col>/</col><col>filesystem</col><col>The root filesystem</col>
+ </row>
+ <row>
+ <col>2</col><col>1</col><col>/home</col><col>filesystem</col><col>Our users' homedirs</col>
+ </row>
+ <row>
+ <col>3</col><col>1</col><col>glibc</col><col>package</col><col>The standard C library</col>
+ </row>
+ <row>
+ <col>4</col><col>1</col><col>arjen</col><col>user</col><col>Arjen Baart</col>
+ </row>
+</table>
+</subsubsection>
+</subsection>
+
+
+<subsection>
+<heading>parameter_class</heading>
+<para>
+Each parameter is defined to be of a certain <emph>class</emph>.
+The class defines which properties a parameter of that class may have.
+</para>
+
+<subsubsection>
+<heading>The fields</heading>
+<para>
+The fields of the <emph>parameter_class</emph> table are listed below:
+</para>
+<table cpos='lllp{6cm}'>
+ <thead>
+ <col> Fieldname </col> <col> Fieldtype </col> <col> Size </col>
+ <col> Remarks </col>
+ </thead>
+ <row>
+ <col>name</col><col>text</col><col> </col>
+ <col>Name of the class</col>
+ </row>
+ <row>
+ <col>property_name</col><col>text</col><col> </col>
+ <col>Name of the property. Used in property table.</col>
+ </row>
+ <row>
+ <col>description</col><col>text</col><col> </col>
+ <col>A verbose description of the property</col>
+ </row>
+ <row>
+ <col>property_type</col><col>text</col><col> </col>
+ <col>Either 'STATIC' or 'DYNAMIC'</col>
</row>
<row>
- <col>
- <para>ous_objectid</para>
- </col>
- <col>
- <para>objectid</para>
- </col>
- <col>
- <para/>
- </col>
+ <col>min</col><col>float</col><col>4</col>
+ <col>The default minimum value of the property.</col>
</row>
<row>
- <col>
- <para>ous_username</para>
- </col>
- <col>
- <para>username</para>
- </col>
- <col>
- <para/>
- </col>
+ <col>max</col><col>float</col><col>4</col>
+ <col>The default maximum value of the property.</col>
</row>
<row>
- <col>
- <para>ous_security_level</para>
- </col>
- <col>
- <para>ous_security_level</para>
- </col>
- <col>
- <para/>
+ <col>notify</col><col>boolean</col><col>1</col>
+ <col>If TRUE, create a notification when something about the property or
+ the parameter changes.
</col>
</row>
- </table>
-</paragraph>
+
+</table>
+
+<para>
+The combination of name and property_name must be unique.
+Note that the <emph>min</emph> and <emph>max</emph> fields are used
+only for properties of a numerical nature.
+</para>
+
+</subsubsection>
+<subsubsection>
+<heading>Sample data</heading>
+
+<para>
+The table below lists a few examples of parameter classes
+</para>
+<table cpos='lllllll'>
+ <thead>
+ <col>name</col><col>property_name</col><col>description</col><col>property_type</col>
+ <col>min</col><col>max</col><col>notify</col>
+ </thead>
+ <row>
+ <col>package</col><col>version</col><col>The installed version</col><col>STATIC</col>
+ <col> </col><col> </col><col>true</col>
+ </row>
+</table>
+</subsubsection>
+</subsection>
-<paragraph>
- <heading>The relationships.</heading>
+<subsection>
+<heading>parameter_notification</heading>
<para>
-Relationshiips with other tables:
+The parameter_notification table defines the relationship between <emph>parameters</emph>
+and <emph>notifications</emph>.
+Whenever a parameter is changed, i.e. the parameter is created, one of its
+properties changed or a parameter is removed, this may result in a notification.
+This table provides the link between that notification and the change in
+parameter that the notification is about.
+Note that a single notification may be created for a number of changes in
+parameters.
</para>
-<table cpos='llp{8cm}'>
+<subsubsection>
+<heading>The fields</heading>
+<para>
+The fields of the <emph>parameter_notification</emph> table are listed below:
+</para>
+<table cpos='lllp{6cm}'>
<thead>
- <col>
- <para>Fieldname</para>
- </col>
- <col>
- <para>Remote Table</para>
- </col>
- <col>
- <para>Remarks</para>
- </col>
+ <col> Fieldname </col> <col> Fieldtype </col> <col> Size </col>
+ <col> Remarks </col>
</thead>
<row>
- <col>
- <para>objectID</para>
- </col>
- <col>
- <para>object</para>
- </col>
- <col>
- <para>Reference to the object (which object does this apply to).</para>
- </col>
+ <col>notificationid</col><col>bigint</col><col>8</col>
+ <col>The notification for the changed parameters. Refers to the notification table.</col>
</row>
<row>
- <col>
- <para>username</para>
- </col>
- <col>
- <para>user</para>
- </col>
- <col>
- <para>Reference to the user.</para>
- </col>
+ <col>paramid</col><col>bigserial</col><col>8</col>
+ <col>The parameter for which the notification is made. Refers to the parameter table.</col>
</row>
- </table>
+</table>
- <para>In the model this look like this:</para>
- <picture src="erd-objusr.png" eps="erd-objusr"/>
+<para>
+The combination of notificationid and paramid must be unique.
+</para>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>priority</heading>
<para>
-The priority table contains information on the levels that are recognized by the system. Mainly data retrieval so depending on indexing. It needs to be said that most likely only a couple of states will exist<footnote>By default we will use five states, but many states can be given to enable all types of differentiation.</footnote>.
+The priority table contains information on the levels that are recognized by the system.
+Mainly data retrieval so depending on indexing.
+It needs to be said that most likely only a couple of states will exist<footnote>By default
+we will use five states, but many states can be given to enable all
+types of differentiation.</footnote>.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
<para>
<picture src="erd-prior.png" eps="erd-prior"/>
</para>
-</paragraph>
</subsubsection>
+</subsection>
+
+<subsection>
+<heading>property</heading>
+<para>
+The property table stores the actual values of the properties of
+operational parameters of a monitored object.
+</para>
+
+<subsubsection>
+<heading>The fields</heading>
+<para>
+The fields of the <emph>property</emph> table are listed below:
+</para>
+<table cpos='lllp{6cm}'>
+ <thead>
+ <col> Fieldname </col> <col> Fieldtype </col> <col> Size </col>
+ <col> Remarks </col>
+ </thead>
+ <row>
+ <col>paramid</col><col>bigint</col><col>8</col>
+ <col>The parameter to which this property belongs. Refers to the parameter table</col>
+ </row>
+ <row>
+ <col>name</col><col>text</col><col> </col>
+ <col>Name of the property</col>
+ </row>
+ <row>
+ <col>value</col><col>text</col><col> </col>
+ <col>The current value of the property</col>
+ </row>
+ <row>
+ <col>type</col><col>enum</col><col> </col>
+ <col>Dynamic or Static</col>
+ </row>
+ <row>
+ <col>minimum</col><col>float</col><col>8</col>
+ <col>The minimum value of the property (for numerical properties only)</col>
+ </row>
+ <row>
+ <col>maximum</col><col>float</col><col>8</col>
+ <col>The maximum value of the property (for numerical properties only)</col>
+ </row>
+
+</table>
+</subsubsection>
<subsubsection>
+<heading>Sample data</heading>
+
+<para>
+The table below lists a few examples of properties
+</para>
+<table cpos='llllll'>
+ <thead>
+ <col>paramid</col><col>name</col><col>value</col>
+ <col>type</col><col>minimum</col><col>maximum</col>
+ </thead>
+ <row>
+ <col>1</col><col>size</col><col>400000</col>
+ <col>STATIC</col><col>100000</col><col>999999999</col>
+ </row>
+ <row>
+ <col>1</col><col>used</col><col>200000</col>
+ <col>DYNAMIC</col><col>50000</col><col>400000</col>
+ </row>
+ <row>
+ <col>2</col><col>size</col><col>3000000</col>
+ <col>STATIC</col><col>100000</col><col>999999999</col>
+ </row>
+ <row>
+ <col>2</col><col>used</col><col>2000000</col>
+ <col>DYNAMIC</col><col>50000</col><col>2700000</col>
+ </row>
+ <row>
+ <col>3</col><col>version</col><col>2.2.5-39</col>
+ <col>STATIC</col><col>0</col><col>0</col>
+ </row>
+</table>
+</subsubsection>
+</subsection>
+
+<subsection>
<heading>service</heading>
<para>
-The table <emph>service</emph> indicates the service that can be handled by the system. Out of the servicelist the administrator can indicate what services to expect.
+The table <emph>service</emph> indicates the service that can be handled by the system.
+Out of the servicelist the administrator can indicate what services to expect.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
<para>4</para>
</col>
<col>
- <para>The advised priority if these log-entries don't come in<footnote>Advised priorities can be changed in the object_service table per object. </footnote>.</para>
+ <para>The advised priority if these log-entries don't
+ come in<footnote>Advised priorities can be changed in the object_service
+ table per object. </footnote>.</para>
</col>
</row>
<row>
<para>4</para>
</col>
<col>
- <para>The maximum priority advised for this service<footnote>Advised priorities can be changed in the object_service table per object.</footnote>.</para>
+ <para>The maximum priority advised for this
+ service<footnote>Advised priorities can be changed in the
+ object_service table per object.</footnote>.</para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
Relationships with other tables:
<para>
<picture src="erd-service.png" eps="erd-service"/>
</para>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>status.</heading>
<para>
-The table <emph>status</emph> contains the possible states that a notification can have. As with the table <emph>priority</emph> these statuses are limited in number by default but can be expanded. Also here retrieval prevails above data entry and therefor indexing is important.
+The table <emph>status</emph> contains the possible states that a notification can have.
+As with the table <emph>priority</emph> these statuses are limited in number by
+default but can be expanded. Also here retrieval prevails above data entry and
+therefor indexing is important.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
The table is indexed on the following fields:
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
Relationships with other tables
<para>
<picture src="erd-status.png" eps="erd-status"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Default values.</heading>
<para>The status values are default for the system and for that reason are predefined.</para>
-<table cpos='lllp{6cm}'>
+<table cpos='p{2cm}lp{1cm}p{6cm}'>
<thead>
<col>
<para>Statuscode</para>
<para>T</para>
</col>
<col>
- <para>The notification has been worked on. After it has been verified the notification can be closed.</para>
+ <para>The notification has been worked on.
+ After it has been verified the notification can be closed.</para>
</col>
</row>
<row>
</col>
</row>
</table>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>type_of_issue.</heading>
-<para>This table will contain a list of all available issues that can be detected. All issues have a suggested priority setting. This is typically data retrieval and good indexing is needed.</para>
+<para>
+This table will contain a list of all available issues that can be detected.
+All issues have a suggested priority setting.
+This is typically data retrieval and good indexing is needed.</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
<para>Is this check currently being used in the system.</para>
</col>
</row>
+ <row>
+ <col><para>automated_check</para></col><col><para>Boolean</para></col><col><para>1</para></col>
+ <col><para>Is this an automated check that can be executed by gcm_daemon</para></col>
+ </row>
+ <row>
+ <col><para>alter_level</para></col><col><para>Integer</para></col><col><para>4</para></col>
+ <col><para>This field indicates what the default priority-level for a notification will be if this issue is raised.</para></col>
+ </row>
+ <row>
+ <col><para>last_run</para></col><col><para>Timestamp</para></col><col><para/></col>
+ <col><para>The time that the last processing took place.</para></col>
+ </row>
+ <row>
+ <col><para>recheck_interval</para></col><col><para>Timestamp</para></col><col><para/></col>
+ <col><para>The interval that is needed before rerunning this check.</para></col>
+ </row>
+
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
The table is indexed on the following fields:
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<table cpos='lll'>
<thead>
<para>
<picture src="erd-toi.png" eps="erd-toi"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Default values.</heading>
<para>
-All checks are entered as code into the system. This table only works for the application only. The user can set specifics in the application only.
+All checks are entered as code into the system.
+This table only works for the application only.
+The user can set specifics in the application only.
</para>
<table cpos='lllll'>
<thead>
</col>
</row>
</table>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>unprocessed_log</heading>
<para>
-The <emph>user</emph> table contains the users that can login the monitoring application. It will also store if the users maintains the system. Mainly used for retrieval so properly indexed.
+The <emph>user</emph> table contains the users that can login the monitoring application.
+It will also store if the users maintains the system.
+Mainly used for retrieval so properly indexed.
</para>
-<paragraph>
+<subsubsection>
<heading>The fields.</heading>
<para>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes.</heading>
<para>
The indices of the table:
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
Relationships with other tables:
<para>
<picture src="erd-unplog.png" eps="erd-unplog"/>
</para>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>Sample values.</heading>
<para>To do</para>
-</paragraph>
</subsubsection>
+</subsection>
-<subsubsection>
+<subsection>
<heading>user_gnucomo</heading>
-<para>The <emph>user</emph> table contains the users that can login the monitoring application. It will also store if the users maintains the system. Mainly used for retrieval so properly indexed.</para>
-<paragraph>
+<para>
+The <emph>user</emph> table contains the users that can login the monitoring application.
+It will also store if the users maintains the system.
+Mainly used for retrieval so properly indexed.
+</para>
+
+<subsubsection>
<heading>The fields.</heading>
+
<para>
The fields are listed in the table below:
</para>
<para/>
</col>
<col>
- <para>Sessionnumber currently used by user. If this is set to 0 a user is not present on the system. Only one session can be open at a time.</para>
+ <para>Sessionnumber currently used by user.
+ If this is set to 0 a user is not present on the system.
+ Only one session can be open at a time.</para>
</col>
</row>
<row>
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The indexes</heading>
<para>
The table is indexed on the following fields:
</col>
</row>
</table>
-</paragraph>
+</subsubsection>
-<paragraph>
+<subsubsection>
<heading>The relationships.</heading>
<para>
Relationships with other tables
<para>
<picture src="erd-usr.png" eps="erd-usr"/>
</para>
-</paragraph>
</subsubsection>
</subsection>
</section>
-<section>
- <heading>Model of table-relations.</heading>
-
-<para>The following model pictures the database as it has been described before.</para>
-<para>
- <picture src="erd.png" eps="erd" scale="0.7"/>
-</para>
-
-</section>
</chapter>
<chapter>
<heading>User Interface.</heading>
<para>To be determined in the near future.</para>
- </chapter>
+</chapter>
- <chapter>
- <heading>The installation process.</heading>
+<chapter>
+ <heading>The installation process.</heading>
+
+<para>
+Since the system must make maintenance and security easier to use,
+the burden of installation should as easy as possible.
+Where possible the installation script should take away as much work as possible.
+Where settings need to be done, this should be done through an interface.
+However at no point we take the user's right to understand and work with system.
+Configuration-files should be easy to understand and the choice must be there to do installation manually.
+For the time being, we will use the manual installation procedure outlined below:
+</para>
+
+<para>
+Since there is no binary package available for Gnucomo yet, you will need
+to compile and install Gnucomo from the source code.
+Before making the Gnucomo binaries, make sure you have the following
+packages installed:
+<itemize>
+<item>postgresql, postgresql-server, postgresql-develop</item>
+<item>AXE</item>
+<item>libxml2, libxml2-develop</item>
+</itemize>
+
+Make sure your PostgreSQL database server is up and running.
+If you also want to use the web interface, you will need Apache with PHP.
+The PHP module needs Postgresql and DOM-XML support.
+With all required packages installed, you should be able to go into
+the <emph>src</emph> directory and type <code>make</code> to create
+a binary <code>gcm_input</code>.
+</para>
+
+<para>
+To use gnucomo, you need to create a database and a configuration file.
+To make the database in your PostgreSQL server, log in as a DBA (DataBase
+Administrator, usually the user 'postgres') and create the database and a
+user who can use the database.
+Here is an example:
+
+<verbatim>
+
+ createdb gnucomo
+ createuser arjen
- <para>Since the system must make maintenance and security easier to use, the burden of installation should as easy as possible. Where possible the installation script should take away as much work as possible. Where settings need to be done, this should be done through an interface. However at no point we take the user's right to understand and work with system. Configuration-files should be easy to understand and the choice must be there to do installation manually. </para>
+</verbatim>
+If you also want to be able to use the test scripts, you will need to
+create the <code>gnucomo_test</code> database as well.
+The configuration file for Gnucomo is a rather simple XML file that
+states at least what database Gnucomo uses and the userid with which
+Gnucomo will log in to the database server.
+These parameters should be the same as the database and user you just
+created in your role of DBA.
+There is an example configuration file, <code>gnucomo.conf</code> in the
+<emph>src</emph> directory.
+You should copy this config file to one of the following places:
+<enumerate>
+<item><code>/etc/gnucomo.conf</code></item>
+<item><code>/usr/local/etc/gnucomo.conf</code></item>
+</enumerate>
+With the database and the configuration file in place, you should
+be able to run <code>gcm_input</code> to read log files and store
+log entries in the database.
+</para>
<section>
<heading>Supported platforms.</heading>
<para>RedHat Linux (.rpm packages)</para>
</item>
</itemize>
- <para>We will try and facilitate as many operating systems client-side and as many unices server-side, but efforts on testing out of the projects will be very minimalistic to ensure that the project keeps delivering new version and new features. </para>
+ <para>
+ We will try and facilitate as many operating systems client-side and
+ as many unices server-side, but efforts on testing out of the projects
+ will be very minimalistic to ensure that the project keeps delivering
+ new version and new features.
+ </para>
</section>
<section>
</section>
</chapter>
- <chapter>
- <heading>Dependency on other free software.</heading>
- <para>The following list is a set of applications that will be used on to make our application work:</para>
+<chapter>
+ <heading>Dependency on other free software.</heading>
+
+<para>The following list is a set of applications that will be used on to make our application work:</para>
<table cpos='lp{8cm}l'>
<thead>
<col>
</col>
</row>
</table>
- <para/>
- </chapter>
+<section>
+<heading>Related projects</heading>
- <chapter>
- <heading>Settings on the server machine.</heading>
+<para>
+There are a number of projects that can help <strong>Gnucomo</strong> or perform
+similar funtions:
- <section>
- <heading>Required.</heading>
+<itemize>
+ <item>
+ <reference href='http://crm114.sourceforge.net/'>CRM114</reference> - The Controllable
+ Regex Mutilator.
+ </item>
+</itemize>
+</para>
+</section>
+</chapter>
+
+<chapter>
+ <heading>Settings on the server machine.</heading>
- <para>The following settings are required to ensure that the functionality is as much as expected.</para>
+<section>
+ <heading>Required.</heading>
- <subsection>
- <heading>Timezone in GMT (UTC).</heading>
+<para>The following settings are required to ensure that the functionality is as much as expected.</para>
- <para>Since all international traffic registers all entries in UTC (Universal Time Coordinate) this system will do so as well. Therefore the clock has to be adjusted to that as well as the system settings.</para>
- <para>These settings can found on a RedHat computer in the <code>/etc/sysconfig/clock-file</code>. On Debian this stored in the file <code>/etc/timezone</code>.</para>
- </subsection>
- </section>
+<subsection>
+ <heading>Timezone in GMT (UTC).</heading>
- <section>
- <heading>Suggested.</heading>
+<para>Since all international traffic registers all entries in UTC (Universal Time Coordinate) this system will do so as well. Therefore the clock has to be adjusted to that as well as the system settings.</para>
+<para>These settings can found on a RedHat computer in the <code>/etc/sysconfig/clock-file</code>. On Debian this stored in the file <code>/etc/timezone</code>.</para>
+</subsection>
+</section>
- <subsection>
- <heading>Use NTP.</heading>
+<section>
+ <heading>Suggested.</heading>
- <para>If a computer is running for some time the clocks tend to be off the correct time. This makes it harder to detect what exactly happened exactly at what moment in time and reduce value of log-entries. Especially considering that ultimately all data is gathered in one central system. To overcome this Network Time Protocol (NTP RFC 13025 March 1992) has been created that explains a protocol to synchronize clocks through the Internet. Many operating systems like Microsoft Windows 2000 Server and FreeBSD, OpenBSD and Linux support this. A ntp-server (or ntpd) can be found at: http://www.eecis.udel.edu/~ntp/ It is strongly recommended that you use this.</para>
+<subsection>
+ <heading>Use NTP.</heading>
+
+<para>If a computer is running for some time the clocks tend to be off the correct time. This makes it harder to detect what exactly happened exactly at what moment in time and reduce value of log-entries. Especially considering that ultimately all data is gathered in one central system. To overcome this Network Time Protocol (NTP RFC 13025 March 1992) has been created that explains a protocol to synchronize clocks through the Internet. Many operating systems like Microsoft Windows 2000 Server and FreeBSD, OpenBSD and Linux support this. A ntp-server (or ntpd) can be found at: http://www.eecis.udel.edu/~ntp/ It is strongly recommended that you use this.</para>
</subsection>
</section>
projects / gnucomo.git / blobdiff