(my ex-employer), we settled on a
very useful combination of free software to help us manage
development. We used
In the software group at (my ex-employer), we settled on a
very useful combination of free software to help us manage
development. We used
We made some small changes to these packages locally to help integrate them into our development process. With these small changes, we had diverse open source tools linked almost seamlessly:
The work we have done here was not rocket science, but was quite fiddly and time-consuming. We hope that others find it useful - please feel free to mail me with comments.
NOTE! I have used example.com throughout
these pages. Be aware that the links I'm using here won't work!
Example.com is reserved for use in documentation and not available for
registration. See RFC 2606, Section 3. You'll (obviously) need to
change config files to make this lot work for you!
CVS is (probably) the world's most commonly-used software for source code control. It is a tried-and-tested stable piece of open source software that has been continuously developed and maintained since 1989.
On our internal network, we use the pserver access
method, as this is most easily configured for the range of client
machines that we commonly use (Linux, FreeBSD and Windows). For
external access, we use the ext method and cvs over ssh -
this is useful as ssh provides a simple secure connection that is both
authenticated and encrypted.
Most of the configuration of CVS is done through the files in the
CVSROOT module, which is itself managed through CVS (of
course!).
The first thing to do is make grab the logs from CVS checkin
messages. To do this, modify the file
CVSROOT/loginfo. The version we use looks (something)
like:
^genstr (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- genstr" developer1 developer2
^twedmon (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- twedmon" developer1 developer2
^udo_demo (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- udo_demo" developer2 developer3
^bugzilla-test (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- bugzilla-test" developer4
^moinmoin (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- moinmoin" developer2 developer1 developer4
CVSROOT (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- CVSROOT" cvs-watchers
DEFAULT (echo ""; echo $USER; date; cat) | mail -s "CVS commit -- Other modules" developer1
ALL (echo ""; echo $USER; date; cat) >> $CVSROOT/CVSROOT/commitlog
ALL (echo ""; echo "Date: "`date`; cat) | $CVSROOT/CVSROOT/bugzilla-watcher $USER %{SVv}
This warrants a little explanation!
DEFAULT matches any commit that has not yet
been matched
ALL is executed for all
checkins.
In the above list, project members are mailed about checkins on the
various modules that they work on (genstr, twedmon, udo_demo,
bugzilla-test and moinmoin). CVSROOT is special, and the
commits are sent to a local mailing list called cvs-watcheers. The
first ALL line simply appends all commit messages to a
logfile that we find helpful for searching.
The last line is most important here. It passes all commit messages
through a locally-written script called
bugzilla-watcher. The variable $USER will
evaluate to the name of the user running the script (i.e. the user
running the checkin). The macro %{SVv} expands to a
comma-separated list of files checked in, of the form:
<DIRECTORY> "<FILE1>",<OLDVER1>,<NEWVER1> "<FILE2>",<OLDVER2>,<NEWVER2> ... "<FILEn>",<OLDVERn>,<NEWVERn>
From this information, the bugzilla-watcher script can (with some work) extract all the information it needs to log all the changes (filename, old version number and new version number) along with their commitlog entries. On additions and removals, the version number shows up as "NONE".
NOTE! The %S entry in the macro is not
standard in CVS! We have added it in to our version of CVS (patch for 1.11 ; patch for 1.12.7). It behaves exactly as
the %s macro, but it adds quotes around the file name
which makes dealing with silly filenames containing spaces
much easier. As I'm also the Debian CVS maintainer, current Debian CVS
packages also have this patch applied. If you can't or don't want to
apply this patch and build your own CVS binary, use %s
instead and modify the bugzilla-watcher script - you'll just have
problems with those silly filenames. See the cvs
documentation for more information about the macros available.
CVSROOT/rcsinfo controls templates used on commits and
imports. The format of the rcsinfo file is similar to the loginfo file
- the first column is a regexp; when it matches the checkin directory,
use the filename on the rest of the line as a template. We have added
a simple single template called rcstemplate; if you want
different messages for different modules, this is easy enough. Our
rcsinfo file looks like:
DEFAULT $CVSROOT/CVSROOT/rcstemplate
CVSROOT/rcstemplate is the simple template that is
given to the user when they are asked for a CVS commit
message. What we use is:
BugID: CVS: Please fill in the above fields with information about your checkin. CVS: Make sure that you know for certain what you are checking in.
As in the default CVS commit message, lines beginning with "CVS:" are
stripped from the message before checkin. We use the
BugID: tag later in the bugzilla-watcher script;
developers are told to add the Bugzilla bug number(s) on that line,
and this is how we track where the commit log messages are meant to
go.
NOTE! If people check in using cvs commit
-m "message", then they will not see this template. Also, the
templates will only be applied to new checkouts; existing checkouts
will not start to use it, even if people cvs update.
CVSROOT/checkoutlist contains a list of files that should
be tracked in the CVSROOT special modules alongside the
files that CVS knows about internally. Ours lists
rcstemplate bugzilla-watcher
so that these local files are tracked too.
CVSROOT/bugzilla-watcher is a locally-written perl
script. It accepts the information about each changed file that is
passed in from CVS loginfo, and reformats that information into
something more useful. It then packages this up into an email which is
sent to the bugzilla system. I didn't write the script, and it's
grotty perl :-). It's here.
This outputs email like:
To: bugzilla-interface@example.com Subject: [Bug 522] CVS Checkin From: CVS User <user@example.com> CVS COMMIT LOG MESSAGE: Make the stripspaces code work in preview mode too. Needs changes to send_editor() - modify the contents of the edit box send_page() - modify the contents of the preview pane do_savepage() - make sure we use the stripspaces option on preview too BRANCH: HEAD FILES CHECKED IN: moinmoin/Page.py 1.12 1.13 moinmoin/PageEditor.py 1.6 1.7 moinmoin/wikiaction.py 1.5 1.6
which is sent to the bugzilla system, ready to be further parsed and dealt with.
Bugzilla is the defect-tracking system, or bug tracker, that was written to support the development of Netscape products and more recently Mozilla. It is a powerful Open Source tool that compares very favourably with the expensive proprietary systems available today. It is especially popular among users and developers of other Open Source projects, but also has a growing following in commercial environments.
Bugzilla is a large perl application that implements a web interface
on top of a backend SQL database. As well as the web frontend that
most users are aware of, there has also been some work done to add an
email interface that can be used to manipulate bugs. In the contrib
directory of the bugzilla distribution, read the file README.Mailif
for instructions on how to set this up. We have done exactly as
specified here, and set up a user account with access to the bugzilla
database. Procmail for that user pipes incoming mail into the
bugzilla_email_append.pl script. We don't use the
bug_email.pl script normally.
For reference, there is a bug open against bugzilla right now that covers the patches here, see http://bugzilla.mozilla.org/show_bug.cgi?id=199116.
We have slightly modified the input script
bugzilla_email_append.pl from the original. We have added
the capability to enter the same commit message against multiple bug
numbers. We also cope with the case where the user checking in does
not have a bugzilla account (e.g. on a build machine with a common
login used by many users). The bugzilla_email_append.pl
will simply append the body of any email it receives to the bug
number(s) specified in the Subject: line, which is quite
straightforward.
Our version of this script is here.
Now that we have CVS commit messages added into the appropriate bug
reports, the logical next step is to link those commit messages back
into CVS itself, so that it is possible to easily view the changes
committed at each step. We have ViewCVS installed
on the CVS server, which is a simple-to-use web front end for CVS. By
making bugzilla parse the comments it prints when viewing bug reports,
we can turn the files and versions in the checkin comments into links
into the CVS repository. The code that talks to the database and turns
bug reports into HTML is in globals.pl. We hook into the
code in quoteUrls() that already parses the comments, and
add a chunk of code that converts the format in the email above into
something like:
CVS COMMIT LOG MESSAGE: Make the stripspaces code work in preview mode too. Needs changes to send_editor() - modify the contents of the edit box send_page() - modify the contents of the preview pane do_savepage() - make sure we use the stripspaces option on preview too BRANCH: HEAD FILES CHECKED IN:
| moinmoin/Page.py | 1.12 | 1.13 | View diff |
| moinmoin/PageEditor.py | 1.6 | 1.7 | View diff |
| moinmoin/wikiaction.py | 1.5 | 1.6 | View diff |
Simply clicking on the links here shows you all the changes that were committed for a given bug. The patch is quite small, but not especially pretty. There are two patches to choose from here:
ViewCVS is a Python
package that allows web access to CVS repositories. Configuration is
quite easy, so I won't cover it here. We have made one simple change to the code - in the code that
converts source and logs into HTML for printing, we have added a new
regexp that matches the BugID tag we use and converts the
bug numbers following it into links back to bugzilla.
HINT:We use several different Apache virtual host
configurations in the system described here. This is thoroughly
recommended, as it makes life much easier for users and admins
alike. Users only have to remember service names (and no trailing
/cgi-bin/foo/bar?code=wibble etc.), and it becomes
possible to rearrange services onto multiple different machines easily
without end-users noticing if real machine names and paths are not
hard-coded everywhere. We currently use one machine to host all of the
services here, but it is aliased as
See the MoinMoin project page for now. I have quite a lot of local changes that I'd like to be merged upstream, but upstream has moved on from the point where I did this work. I'll need to find some time to merge some of it forward and push it upstream.
Steve McIntyre
steve@einval.com
"Can't keep my eyes from the circling sky,
Tongue-tied & twisted, Just an earth-bound misfit, I..."