.\"
.\"     aegis - project change supervisor
.\"     Copyright (C) 1999, 2005-2008, 2010, 2012 Peter Miller
.\"
.\"     This program is free software; you can redistribute it and/or modify
.\"     it under the terms of the GNU General Public License as published by
.\"     the Free Software Foundation; either version 3 of the License, or
.\"     (at your option) any later version.
.\"
.\"     This program is distributed in the hope that it will be useful,
.\"     but WITHOUT ANY WARRANTY; without even the implied warranty of
.\"     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
.\"     GNU General Public License for more details.
.\"
.\"     You should have received a copy of the GNU General Public License
.\"     along with this program. If not, see
.\"     <http://www.gnu.org/licenses/>.
.\"
.bp
.nh 2 "Detecting History File Corruption"
.LP
When you have files which exist for long periods of time, particularly
files such as the ones typically used by history tools, which are
generally appended to, without modification of the bulk of the file, there
is a very real possibility that a block of the file could become corrupted
over the years.\**
.FS
See also
Saltzer, J.H. \fIet al\fP (1981) \f[I]End\[hy]to\[hy]end arguments in
system design\fP,
http://web.mit.edu/\%Saltzer/\%www/\%publications/\%endtoend/\%endtoend.pdf
.FE
Unless you access the file versions contained within that
block, you have no way of knowing whether or not the history file is OK.
(Arguably, the operating system should check for this, but many do not,
and in any case the error may not be detectable at that level.)
.LP
Using Aegis, you can add a simple checksum to your history files which
will detect many cases of corruption such as this, for all of the commonly
used history tools.  Note: it cannot detect all corruptions (nothing can)
but it will detect more than many operating systems will.
.LP
You don't need to use this technique with SCCS or \fIaesvt\fP(1),
they already have checksums in their files.
.nh 3 "General Method"
.LP
In general, you need to do three things:
.IP 1. 3n
You need to create some kind of checksum of your history file each time
you modify it.  Something like \fImd5sum\fP(1) from the GNU Fileutils
would be good.  Store the checksum in a file next to the history file.
This would be done in the
\fIhistory_\%create_\%command\fP and
\fIhistory_\%put_\%command\fP fields of the project \fIaegis.conf\fP file.
.IP 2. 3n
Each time the file is read, you need to verify the file's checksum.
Use the same checksum utility as before, and then compare it using, say,
\fIcmp\fP(1); it it fails (either an IO error, or the checksum doesn't
compare equal) then don't proceed with the history file access.  You may
need to repair or replace the disk.  You will need to restore from backup
(yesterday's backup, see below).  This would be done at the beginning
of the \fIhistory_\%create_\%command\fP, \fIhistory_\%put_\%command\fP,
\fIhistory_\%get_\%command\fP and \fIhistory_\%query_\%command\fP fields
of the project \fIaegis.conf\fP file.
.IP 3. 3n
Because you may not actually interact with the file for years at a time,
you need to check the file fingerprints much more often.  Daily or at
least weekly is suggested.  You do this with a \fIcron\fP(1) job run
nightly which compares all of the history files with their \fImd5sum\fP(1)
checksums.  Email failures to the system administrator and the project
administrators.  By doing this nightly, you not only avoid backing\[hy]up
corrupted files, you will always know on which backup tape the good copy
resides \- yesterday's.
.nh 3 "Configuration Commands"
.LP
In order to implement this,
you need to modify some fields of your project \fIaegis.conf\fP file as follows:
.IP history_create_command 2m
You need to test if the history file and its checksum file exist, and
check the checksum if this is the case.  Then, use whichever history tool
you choose (see the previous sections of this chapter).  If it succeeds,
run \fImd5sum\fP(1) over the history file (\fInot\fP the source file)
and store the checksum in a file next to the history tool's file.
Using the same filename plus a \f[CW]\&.md5sum\fP extension makes the
\fIcron\fP(1) job easier to write.
.IP history_put_command 2m
You need to test if the file exists (it may, for example, be an old
project to which you have recently added this technique) and check the
checksum if this is the case.  Then, use your history tool as normal.
If it succeeds, run \fImd5sum\fP(1) over the history file (\fInot\fP
the source file) as in the create case.
.IP history_get_command 2m
You need to test if the file exists (it may, for example, be an old
project to which you have recently added this technique) and check the
checksum if this is the case.  Then use your history tool as normal.
.IP history_head_command 2m
This command is only used at \fIaeipass\fP file, immediately after one
of the \fIhistory_\%create_\%command\fP or \fIhistory_\%put_\%command\fP
commands.  It is up to you whether you think you need to add a guard as
for the \fIhistory_\%get_\%command field.
.nh 3 "An Alternative"
.LP
Rather than run \fImd5sum\fP(1) on the history files each time
you modify them, you could use \fIgzip\fP(1) to obtain some minor
compression, but it also provides and Adler32 checksum of the file.
For files with long histories, this can be tedious to unpack every time
you need to extract an old version, but such operations are frequently
I/O bound, and so there may be no perceived slowness by the user..
.nh 3 "Aegis' Database"
.LP
In addition to your history files, Aegis maintains a database of file
meta\[hy]data.  In order to add a checksum to the various file making up
the database, turn on the \fIcompressed_database\fP project attribute.
In addition to compressing the database (a minor savings) it also adds
an Adler32 checksum.
.LP
You can check this in the \fIcron\fP(1) job by using \fIgzcat\fP(1)
sent to \fI/dev/null\fP.
.\" vim: set ts=8 sw=4 et :