This document (Publishing this document) is for Savannah administrators, not Savannah users. It describes the GNU Savannah software and takes the subversions installation as example. Savannah is a SourceForge clone based on the SourceForge-2.0 software. It is dedicated to Free Software projects.
Because of the highly specific nature of the software, Savannah is a fork of the SourceForge-2.0 software. Attempting to make it modular and configurable can be seen as a waste of time. Anyway, Jaime E. Villate and Mathieu Roy are trying to do improve configurability because the original SourceForge software is no longer maintained. The whole Savannah software is available from CVS and is managed by the Savannah project. The ChangeLog explains the modifications made to the original code.
The GNU Savannah Project have two major focus:
--- The Detailed Node Listing ---
Installation
Database
Savannah Administrator
Group Types
Site Specific Content
CVS repositories
Mailing lists for non-GNU packages
System Administration
Savannah currently provides a CVS frontend, web space management thru CVS, HTTP download area, bug tracking and mailing lists.
Setting up Savannah is not an easy task because it has to integrate existing habits and projects without breaking anything. However, the SourceForge Installation Guide by Guillaume Morin helps a lot understanding the software.
First, you need to get a copy of Savannah. See http://savannah.gnu.org/cvs/?group=savannah. The Savannah CVS have several modules. The base module is savannah
# anonymous access cvs -d:pserver:[email protected]:/cvsroot/savannah login cvs -z3 -d:pserver:[email protected]:/cvsroot/savannah co savannah
Then, you can configure it. There are two configuration files. The first one is called local.inc.pl, coded in Perl, the second is local.inc, in PHP, used by the PHP Engine. The second is generated by reading the first, so it does not make sense to modify the second.
To create local.inc.pl, you must use the script sv_update_conf. This is how you should proceed:
cd where_savannah_was_downloaded ./configure make conf
At this point, you should manually edit this file, especially if you want extra features such as kerberos, webalizer (described below). After editing sv_update_conf, you must run sv_update_conf. This is how you should proceed:
# pwd == where_savannah_was_downloaded # Edit the file with the extensible One emacs etc/savannah/local.inc.pl make update-conf
sv_update_conf also created a log and crontab entries. We'll explain this later.
Finish the installation by copying the savannah scripts (with sv_ prefix) typing:
make install
The scripts will be installed in bindir (check the Makefile for the actual value of bindir).
Finally, you need to edit the Apache configuration. Add something like the following (assuming that your configuration file (local.inc.pl) is in /etc/savannah):
<Location thisdomain>
SetEnv SV_LOCAL_INC_PREFIX /etc/savannah
</Location>
Savannah is installed on the machine subversions.gnu.org. The root of the installation is in /subversions/sourceforge. All the software that is not system wide and is needed to run Savannah is installed in this directory. The structure of this directory is similar to FHS-2.1. In the following table the path names are relative to the installation root. All directories covered by the SourceForge Installation Guide are omitted.
/etc/savannah
local.inc.pl and local.inc.
tmp
src/savannah
src/savannah/www
src/savannah/gnuscripts
The whole Savannah software is available from CVS and is managed by the Savannah project.
In the root user home, symlinks exists to the savannah installation. ~/sv_subversions point to /subversions/sourceforge/src/savannah.
GNU Savannah uses MySQL. We plan to make it support PostgreSQL in the future.
The database may be called savannah. On subversions, for historical reasons, it's name is sourceforge.
You may create a ~/.my.cnf for the root user to defines the user/passwd. This is not mandatory but it saves typing them on the command line and allows cron jobs to run. This is an example of this file
[client] user=mysql_user password=hey_this_is_supposed_to_be_secret
To create your database, you must have previously created a basic configuration. See the previous chapter. In the following manners, you can create and initialize the whole database:
make database make database-structure make database-initvalues
You can take a look at savannah/db/mysql, each table
is described here and in the READMEs.
A read-only access to the sourceforge database on subversions is granted to the following machines:
fr.fsf.org
The sv_xml script builds daily an XML dump of the
public information from the Savannah database into a filled called
savannah.xml.
In addition a dump containing information that users may not want to
publish to the public such as email and ssh public keys is built in
/subversions/sourceforge/dumps/savannah.xml. The command line
sv_xml --private
is used to generate this dump.
The directory where those files are stored is defined in local.inc.pl. It's the sys_miscdir.
A set of XSLT files can be written in the /subversions/sourceforge/dumps (in the case of subversions - replace this by your sys_miscdir)
directory to build custom files from the savannah.xml file that
is located in the same directory. This is used, for instance, for
account creation information files. If an XSLT file is created
(a.xsl for instance) the Makefile must be updated to add the
a.txt file in the list of dependencies of the all goal.
For instance:
all: accounts-fsffr.txt accounts.txt myown.txt
The generation of both savannah.xml files and the XSLT processing is run daily from the crontab.
The MySQL database (named sourceforge on subversions) that holds all the
information used by you Savannah installation is dumped daily.
For subversions, see http://savannah.gnu.org/projects/sysadmin/ to find out where the dumps are stored. The dumps are compressed and rotated daily with a maximum of 30, as described in logrotate.conf. The
sv_backup script takes care of all this and is called from the
crontab.
Normally, it's stored in the sys_miscdir defined in the configuration file local.inc.pl.
We recently (Aug 2002) changed the usage of the group_type table. If
you have a old savannah installation, drop the old table and create
the new one by using the files savannah/db/mysql/table_group_type*.
The new group_type usage is described in a whole chapter. See below.
The tables people_skill and people_skill_level are loaded
from the skill database maintained by CJN (http://cjn.sourceforge.net/).
The script sv_skill
loads the XML skill files from CJN and replace the content of the
tables in the savannah database.
You can remove some softwares you do not to be shown on the list: add them to the %ignore table in the sv_skill script and re-run it.
In the case of subversion, we do not want to see proprietary softwares.
cd /subversions/sourceforge/src/savannah/gnuscripts edit sv_skill sv_skill cvs commit -m 'Ignore proprietary software xxxx'
At a given time only one person can be in charge of approving or rejecting projects submitted to the Savannah installation. The /admin/ interface is not fit for concurrent access.
You must create an admin user. Go to the web interface and register an user (should be #101).
Then, execute the following mysql command:
mysql -u user -ppassword -e "insert into user_group (user_id,group_id,admin_flags) values (101,1,'A')" dbname
You can give the admin rights to others users by adapting this command.
Since at a given time only one person can be in charge of approving or rejecting projects submitted to the Savannah installation, the user assigned to this task is the one in charge. It is his responsibility to find someone else before leaving :-)
People currently (Aug 2002) playing this role are Jaime E. Villate, Loic Dachary and Mathieu Roy.
The tasks submission moderation and bug dispatching are assigned to the person currently in charge of approving the projects submitted to Savannah.
The password of the admin user is known by
Loic Dachary, Guillaume Morin, Hugo Gayosso,
Jaime E. Villate and
Mathieu Roy.
Something new we introduced recently (Aug 2002) in Savannah is group type. While, with the original SF code, group type was things like Foundries or Project, things really differents, now it just define thing that can be exactly the same: projects.
It defines what features a group can use (as cvs, mailing-list) and how (mailing-list default address, viewcvs base url, cvsroot directory, base host).
After creating an admin account, you must login and create at least one group type. Choose Group Type Editing in the menu.
While people will register project, they will be asked to tell which type of project their project should be.
On Savannah, we host GNU and non-GNU project. We consider them as group type and so every definitions related to them are stored in group_type table.
The base host is different. While GNU project have savannah.gnu.org as base host, non-GNU have freesoftware.fsf.org.
Configuration files for GNU are in /etc/savannah/savannah-conf and the ones for non-GNU are in /etc/savannah/freesoftware-conf.
Content files for GNU are in /etc/savannah/savannah-content and the ones for non-GNU are in /etc/savannah/freesoftware-content.
Configuration files directly in /etc/savannah are symlinks to /etc/savannah-conf since savannah.gnu.org is the default virtual server.
Apache conf must be edited to reflect this. For example, we should have something like the following for each virtual server.
We also have a special group type called www.gnu.org which is about
www.gnu.org webmastering. On this one, the homepage dir is not guessed
(group type option). It permits to set this dir with a name different
than the group name. For instance, the HTML cvs directory of
japanesegnui is /japan.
<Location thisdomain>
SetEnv SV_LOCAL_INC_PREFIX /etc/savannah-conf
</Location>
You can change content of your Savannah installation. For instance, the footer of every page.
Data is partially separated for the PHP Engine. The default content is in etc/site-specific-content. But you can choose the directory you want and, as you may guess if you've read the previous chapter, choose different content for each group type.
Since we always want to manage content via CVS, the site specific content is in two cvs module called savannah-content and freesoftware-content created following the example of savannah/etc/site-specific-content.
Here, we describe the subversion case. We recommand you to only accept secured transfer protocols. Do you need more informations?
Each project registered on Savannah that is not part of the GNU
project is granted a publicly available file download area at
http://freesoftware.fsf.org/download/projectname/
Each project member can upload files to this directory by using scp or
rsync over ssh to the following location:
freesoftware.fsf.org:/upload/projectname/
Sample commands for doing this are:
# # Copy an entire tree verbatim, that may imply to remove files on # freesoftware.fsf.org. # rsync --delete -av --rsh=ssh . freesoftware.fsf.org:/upload/projectname # # Copy a single file with scp # scp -q file.tar.gz freesoftware.fsf.org:/upload/projectname # # Copy a single file with rsync over ssh # rsync --rsh=ssh file.tar.gz freesoftware.fsf.org:/upload/projectname
A reminder is included in the Project Admin page of each project.
Here, we describe the example of subversions.
For each project registered on Savannah there may be two CVS repositories.
One to store the sources of the project and one to store the web of the
project. The sources repository is in /cvsroot and the
web repository is in /webcvs. This values are defined by Group Type.
Existing projects that migrate to Savannah may want their CVS repository to be transfered to subversions. Time is essential for such an operation since the project contributors want to work on the new repository on subversions and stop using the old. When the author asks [email protected], ask him to send the tarbal by mail or send a URL from which it can be downloaded. Make an appointment with him and guarantee that the repository will be untared on subversions with 24 hours maximum. The project contributor must first create a project on subversions. When you have the tarbal untar it at /cvsroot/project. Make sure it does not contain a CVSROOT that would override the existing CVSROOT. If it does manually copy the history and val-tags files only. Make sure the imported repository is untared under /cvsroot/project/project and does not pollute the root of the repository.
When a project has a license that is not website a source
repository is created under /cvsroot/project with
a private CVSROOT that only contains anoncvs. The developers of the
project have write access to the CVSROOT directory.
This will change soon.
The group project is created to grant write access to the repository
to all the members of the project.
When a Savannah project is assigned the website license, it only
has a portion of the webcvs repository and no source CVS repository.
If the html_cvs field for a given Savannah project is empty, it
is not associated with a part of the webcvs repository.
For compatibility with the cvs setup before Savannah was introduced, /subversions/cvs/common contains symbolic links to the corresponding Savannah directory. Developers are encourage to stop using this historical setup.
The /cvs symbolic link points to /subversions/cvs/common so that people already using it to access their repositories can continue to do so. Before Savannah existed a pserver access was available and Savannah continues to maintain it for these projects, updating the CVSROOT/passwd files with user/password pairs that are in the Savannah database.
In any large project, keeping track of changes is difficult. CVS does a reasonable job of allowing source changes to be controlled and managed, but does not provide tools to make it easier to work with a changing code base. The hardest part of working on a dynamic project with many changing modules is knowing when changes occur, and what those changes are.
Software developers often are heavy email users, spending huge amounts of time working with their email software. Free software developers are among the most serious email addicts out there, sorting through hundreds of emails a day, since this is often the only way to stay in touch with users and fellow developers.
A project member may add e-mail notification of any commits to the
repository by doing the following:
(replace project with the name of the project and user by
the Savannah login name).
cvs -d [email protected]:/cvsroot/project co CVSROOT
In CVSROOT/commitinfo
^project /usr/local/bin/commit_prep -T project -r
In CVSROOT/loginfo
^project /usr/local/bin/log_accum -T project -C -m [email protected] -s %{sVv}
The email address must exist, it will not be automatically generated.
The sv_backup script builds tarbals for each repository in the /cvsroot directory. Those tarbals are stored in the /subversions/cvs/software.backups directory and linked with the savannah.gnu.org:/cvs.backups URL. The tarbals are generated daily, if at least one file in the repository is more recent than the tarbal.
When a project has an html_cvs field that is not empty in the
group table, a web repository is created in
/webcvs/html_cvs. By default the html_cvs field has the
value /software/project/ but it may be edited by the Savannah
administrators at the savannah.gnu.org/admin/ URL. Project members may
not change this value. See the gnujobs and gdb projects for examples.
If a project is tagged as non-gnu (gnu field in table groups set to N) it is given a space in the /non-gnu/project directory instead.
When a Savannah project is assigned the website license, it only
has a portion of the webcvs repository and no source CVS repository.
If the html_cvs field for a given Savannah project is empty, it
is not associated with a part of the webcvs repository.
The group webproject is created to grant write access to the repository
to all the members of the project.
All the www.gnu.org web was imported in /webcvs (early 2001).
When a project is registered on Savannah and there already exists
a directory for it in the repository (either .../software/project or
the value of the html_cvs field), a chgrp -R webproject is done
on this repository to grant the members of the project a write access
to this portion of the web repository and only this one.
The www project in Savannah is treated in a special way. All the
members of the www project have access to the whole repository
in /webcvs. It means that they are always included in every webproject
created. However, the non-gnu projects and not included and webmasters may
not modify them if they are not a member of the group.
Since CVS is not able to handle symbolic links, a simple mechanism has been implemented on the machine hosting the www.gnu.org to allow webmasters to control the symbolic link from the CVS tree.
The special file .symlinks contains a list of file name pairs,
one per line. For instance:
foo.html index.html bar.html other.html
is a valid .symlinks file. Every night a script reads all the
.symlinks files, prepend a ln -s in front of each line
and execute them. Well, in reality it's not that simple but you get
the idea. The .symlinks file can only be used to control the
symbolic link in the directory where they are.
The /webcvs/CVSROOT/loginfo file contains a trigger that update the gnudist.gnu.org:/home/www/html directory whenever a commit is done. There is a single CVSROOT for all the projects that have a web repository.
The /subversions/sourceforge/src/savannah/gnuscripts/sv_www_sync.c program was derived from the /usr/local/bin/webcvs.c program (now obsolete). It is called on each commit to keep the www.gnu.org web site in sync with the CVS repository.
The idea is to runs a cvs update -l (to prevent recursion) in the directory where the commit was done. Since the command will be called once for each directory where a commit did some action there is no need for recursion.
The %{s} argument given in the loginfo file is a single argument that lists the directory and all the files involved. As a special case if the directory was added the file list is replaced by '- New directory'. This is lame since adding the files -, New and directory will produce the same effect, but it's unlikely.
There are three cases to take in account:
In order to prevent security compromision the directory name is quoted.
The traces of all the updates are kept in /var/log/sv_sync_www.log.
The special project www have write access to all the /webcvs
repository. It is possible to create projects that will limit write
access of the members of the project to a subdirectory of the /webcvs
repository only. For instance the bravegw Savannah project only
give write access to the /webcvs/brave-gnu-world part of the repository.
A project bound to a specific subdirectory will grant write access to all the tree under this subdirectory. There is no way, for instance, to grant write access to group B to /webcvs/thispart and write access to group A to /webcvs/thispart/subdir. If you do this group B win and will have write access to /webcvs/thispart recursively and group A will have access to nothing. If you see a way to overcome this limitation, let us know.
The sv_www script generates the map that is published at www.gnu.org to Savannah. It writes the file in /subversions/sourceforge/src/server/standards and commits it. The server/standards directory is a read-write checkout of the www.gnu.org web CVS. The sv_www script is run once a day by the crontab.
A more webmaster oriented documentation explains the organisation of the www.gnu.org CVS tree and the rationale of its usage.
If you haven't already created a mailing list to handle messages sent by CVS, follow these instructions to do so.
To get started, surf to your project's "Project Page." If the "Public
Areas" section of the page doesn't list "Mailing Lists," click over to
the Project Admin page, the to the Edit Public Info
page. Make sure the "Use Mailing Lists:" checkbox is on, and click
"Update." Now go back to your "Project Page."
Go to the Mailing Lists page and click through to the
Admin page. Select Add Mailing List to get a really
easy-to-use form that asks you only two questions:
What do you want to name your list?
myproject-commits, or myproject-checkins?" You can
choose other names, but those are well-recognized by active Free
Software developers.
Should I let just anyone subscribe?
Ok, so you really only had to decide one thing - that's even better. Now, click the "Add This List" button, and wait for your list to be created.
It is convenient to use Savannah to manage accounts on a machine that is
completely unrelated to Savannah itself. For instance, the project
fsffr lists all the
users who should have an account on the france.fsfeurope.org
machine.
A cron job on the remote machine can fetch the list of users from Savannah and update the password files accordingly. Adding a user to the machine can then be done by adding the user as a developer of the project.
A guide to install the savannahusers script on the target
machine is available in the savannahusers manual page. This chapter deals with the necessary
actions on the savannah.gnu.org machine, not on the target machine.
In order for remote machines to take advantage of Savannah for account management, a list of all Savannah users is dumped daily, both in XML format and text format (XML Dump).
The access to the user information is restricted and has to be done
in the following way:
rsync --rsh=ssh [email protected]: .
The user xmlbase on savannah.gnu.org is only used for this
purpose. The ssh public key of the user doing the rsync on the
remote machine must be registered in the authorized_keys file of
the xmlbase. He will only be allowed to access a single file.
You don't need to give the command you want to execute, indeed
this information is already in the authorized_key :
command="rsync
--server --sender . /subversions/sourceforge/dumps/savannah.xml"
1024 35 1325...
Two files may be accessed in this way:
savannah.xml
accounts.txt
loic Loic Dachary [email protected] 1024 35 14482406825620879676223610524821306708503540742800... rodolphe Rodolphe Quiedeville [email protected] 1024 35 13773675641076158303518150007131532895996406770957... 1024 35 13392800240284295490871092259529193810644583890958...
Each account block is separated by an empty line. The first line is
the uniq user name. The second line is the full name of the user. The
third line is the e-mail address of the user. The next lines are the
content of the authorized_keys file.
It is possible to generate files specific to a given target machine.
For instance the account-fsffr.txt file is a
selection of the users that are members of the
fsffr projects. The
Makefile in the dumps directory is responsible for the creation of
these files. It uses XSLT to select the relevant informations from the
savannah.xml dump.
It is rarely needed to remove a user. Inactive accounts on Savannah do not hurt (neither in space nor in performance). When it appears to be necessary to remove a user account, the administrator should proceed as follows:
See Savannah Elisp to get shortcuts, under GNU Emacs, to perform this kind of tasks.
Send all questions and requests to [email protected] or to support requests.
Savannah features a way to link a project with its mailing lists. They are handled by Mailman on fencepost. The purpose of this section is to explain the link between savannah and Mailman.
Some details regarding the setup of Mailman can be found in the sysadmin.texi file at http://savannah.gnu.org/projects/sysadmin/.
Before Savannah was available, some mailing lists have been created. Some of the GNU packages have been migrated on savannah since then. From the list administration page on each Savannah project, it is possible to to make the link between these packages and their mailing lists (the file is savannah.gnu.org/www/mail/admin/index.php). The administrator of the Savannah project has to fill aform with the name of its mailing lists and the admin password of the list.
When a Savannah project administrator chooses to add a mailing list for
his(her) project, an entry is added to the Savannah database. This
information will be dumped by (by the sv_xml script). A cronjob on
fencepost.gnu.org will read that dump and find which lists must be
created. It will launch the newlist binary and update the alias file.
This cronjob that creates mailing lists can be found in gnuscripts/Mailman/mailing_lists_create.pl in the CVS tree (see http://savannah.gnu.org/projects/savannah/ for get the source tree).
The parameters needed to bind an existing mailing list to a Savannah project (list name and password) are checked by a CGI script installed on fencepost.gnu.org. The related files are in the gnuscripts/Mailman directory of the Savannah sources (see http://savannah.gnu.org/projects/savannah/ for get them). They can be installed on fencepost via a Makefile (details are in sysadmin.texi).
The mailing lists of the Savannah projects that are not part of the GNU project are hosted under the domain freesoftware.fsf.org. This domain and the corresponding mailman installation are installed on the fencepost.gnu.org machine.
Mailman was installed in /com/mailer/freesoftware.
Once more, the Group Type configuration is important for mailing-list configurations.
Savannah will try to send mail to users under various circumstances (bug
reports notification, account creation etc.). In some cases it will use
the real mail address of the user, in others it will use
[email protected]. In order for the [email protected] address
to work properly for outgoing mails, the /etc/email-addresses file is updated
automatically every 5 minutes with the following command:
sv_aliases
The [email protected] can never be used to receive mail for
the good reason that savannah.gnu.org does not listen on the SMTP port.
People who have a simple alias [email protected] but no account on Kerberos
cannot create an account on Savannah. When they ask to unlock the
account name on [email protected], tell them to create an account
using a fake username and to send this username to [email protected].
When receiving that user name substitute the fake login name by the desired
one:
mysql -e "update user set user_name = 'desired' where user_name = 'fake'" sourceforge
Must be root to run this script and make sure you export CVS_RSH=ssh. You are advised to run it in /subversions/sourceforge/tmp, although it is not mandatory.
It is run from the crontab and the output is logged in /var/log/sv_cvs.log.
The sv_cvs perl script generates a shell script that will synchronize
the system files with the state of the Savannah database (sourceforge).
This script only generates lines if something needs to be done. When the
resulting script is executed, another run must not display any
action (unless the database was modified in the meantime which is unlikely).
It performs the following tasks, in this order.
Add new projects
Update existing projects
Add missing users
Remove users
Update existing users
Update the groups of anoncvs
Update the CVS password file
Create download area for non-GNU projects
Update cvs-pserver.conf
The HTML version of this document is published in two places:
Savannah Administration Guide and
Savannah Administration Guide. The source is stored in the
subversions.gnu.org:/cvsroot/savannah CVS repository, in the
$Source: /webcvs/www/www/server/standards/README.savannah.html,v $ file (and/or subversions.gnu.org:/cvs co gnudocs
repository ? - FIXME). To facilitate the publication process you can edit it in
the subversions.gnu.org:/subversions/sourceforge/src/gnudocs directory
and then issue a
make publish
The publish goal assumes that the Savannah document root is in
../savannah/www and a read-write checkout of the www.gnu.org/server/standards
directory is in ../server/standards. It will format the document to
HTML and commit the changes to the repository.
The SSL certificate for savannah.gnu.org was generated in /etc/apache-ssl/. Check the README file for a log of the command. There has been a lot of discussions regarding the root certificate for GNU, the use of a PKI. At some point the savannah certificate will be generated using a proper root certificate.
Project registration that are disapproved with the PHP interface are marked with D in the SQL database.
To remove a project registration completely, sv_register_discard is used, specifying the project unix name as argument. It will ask you to give a comment, an explanation about the discard.
sv_register_discard project_unix_name
Additionally, you can use -user and -comment options:
sv_register_discard project_unix_name --user="yp" --comment="this project does not fit to our policy"
With no argument given, sv_register_discard will search any project marked as D and will ask you for a comment for each project registration discarded.
So, if needed, savannah-hackers can check the reason why a project has been discarded since it is logged in /var/log/sv_register_discard.log
Statistics for savannah.gnu.org web usage are generated using webalizer.
The sv_stat script used to do the job on a daily basis (called from the
crontab) using the webalizer.conf in the
/subversions/sourceforge/src/savannah/www/webalizer directory and moving
the generated report in the same directory.
The sv_stat script is also called before rotating logs, as specified
in the /etc/apache-ssl/cron.conf script.
Now, since we found more interesting to let users choose features they want to use, webalizer must be installed via the normal way: packaged. On subversions.gnu.org, webalizer has been installed via apt-get.
In /etc/savannah/local.inc.pl, Savannah is configured to use webalizer.
The file used is now, as it should be,
/etc/cron.daily/webalizer.
If for a reason of another all the logs need to be regenerated, try:
cd /var/log/apache-ssl for log in `ls --sort=t -r access*`; do echo "parse $log" && webalizer -q -c /usr/local/sourceforge/src/savannah/www/webalizer/webalizer.conf $log; done
Exactly the same goes for mrtg.
The Savannah crontab jobs are in /etc/cron.d/savannah. Every cron command
output is sent to [email protected].
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/subversions/sourceforge/bin:/usr/local/mysql/bin [email protected] # # Build /etc/aliases, # http://savannah.gnu.org/savannah.html#Mails%20and%20aliases # */5 * * * * root sv_aliases # # Build www map, # http://savannah.gnu.org/savannah.html#Web%20CVS%20and%20Projects # 10 4 * * * root sv_www # # Sync projects with CVS related system files, # http://savannah.gnu.org/savannah.html#Users%20and%20CVS%20synchronization # 17 * * * * root cd /subversions/sourceforge/tmp ; sv_cvs | ( date ; sh -x ) >> /var/log/sv_cvs.log 2>&1 # # Daily backups of the Savannah database, # http://savannah.gnu.org/savannah.html#Database%20Backups # 7 5 * * * root sv_backup # # Daily XML dump of Savannah public information # http://savannah.gnu.org/savannah.html#XML%20Dump # 7 6 * * * root sv_xml > /subversions/sourceforge/src/savannah/www/savannah.xml 14 6 * * * root sv_xml --private > /subversions/sourceforge/dumps/savannah.xml 30 6 * * * root make -s -C /subversions/sourceforge/dumps all 0 */1 * * * root sv_xml --list > /subversions/sourceforge/dumps/list.xml # # Daily web server statistics: It remains here for the memory. # As explained in # http://savannah.gnu.org/savannah.html#Web%20Usage%20Statistics # it's now useless and shouldn't be used. # #7 7 * * * root sv_stat
The logs of the Savannah scripts are in /var/log. They are
rotated by the /etc/logrotate.d/savannah configuration file of
logrotate.
/var/log/sv_cvs.log
/var/log/sv_sync_www.log
must be read-write for everyone.
/var/log/apache-ssl/access.log
All software that is not system wide but only used for the purpose of Savannah must be installed in the prefix /subversions/sourceforge.
The MySQL installation is an exception that must be fixed. It is installed with the /usr/local/mysql prefix. It was not installed from the debian package because I ([email protected]) was not able to fix the MySQL-3.23 package to make it work on potato. Now that the machine runs woody, this historical hack should be removed.
The large number of groups a user can have (>32) implies to modify some basic programs (namely useradd and usermod).
The cvs-1.11.1p1 executable was patched and the patches are available at cvs-1.11.1p1-2001-12-11.patch.
Here is the patch applied to /usr/local/src/shadow-19990827. The
modified usermod and useradd have been installed in
/subversions/sourceforge/bin.
*** ./debian/rules.~1~ Fri Feb 9 02:05:06 2001
--- ./debian/rules Fri Feb 9 02:05:41 2001
***************
*** 38,44 ****
ifneq ($(DEB_HOST_GNU_SYSTEM),gnu)
include debian/scripts/login.mk
package-list += binary-login
! config_options += --with-libpam
control_defs += -DMODDEP="(>= 0.72-5)"
endif
--- 38,44 ----
ifneq ($(DEB_HOST_GNU_SYSTEM),gnu)
include debian/scripts/login.mk
package-list += binary-login
! # config_options += --with-libpam
control_defs += -DMODDEP="(>= 0.72-5)"
endif
*** ./build-tree/shadow-19990827/libmisc/addgrps.c.~1~ Mon Dec 28 12:34:41 1998
--- ./build-tree/shadow-19990827/libmisc/addgrps.c Fri Feb 9 03:04:47 2001
***************
*** 20,25 ****
--- 20,28 ----
* already there. Warning: uses strtok().
*/
+ #undef NGROUPS_MAX
+ #define NGROUPS_MAX 512
+
int
add_groups(const char *list)
{
*** ./build-tree/shadow-19990827/src/usermod.c.~1~ Fri Jul 9 09:27:38 1999
--- ./build-tree/shadow-19990827/src/usermod.c Fri Feb 9 03:05:52 2001
***************
*** 74,79 ****
--- 74,82 ----
#define VALID(s) (strcspn (s, ":\n") == strlen (s))
+ #undef NGROUPS_MAX
+ #define NGROUPS_MAX 512
+
static char *user_name;
static char *user_newname;
static char *user_pass;
*** ./build-tree/shadow-19990827/src/groups.c.~1~ Mon Jun 7 09:40:45 1999
--- ./build-tree/shadow-19990827/src/groups.c Fri Feb 9 03:15:54 2001
***************
*** 42,47 ****
--- 42,50 ----
static void print_groups P_((const char *));
int main P_((int, char **));
+ #undef NGROUPS_MAX
+ #define NGROUPS_MAX 512
+
/*
* print_groups - print the groups which the named user is a member of
*
*** ./build-tree/shadow-19990827/src/id.c.~1~ Mon Jun 7 09:40:45 1999
--- ./build-tree/shadow-19990827/src/id.c Fri Feb 9 03:16:34 2001
***************
*** 50,55 ****
--- 50,58 ----
static void usage P_((void));
int main P_((int, char **));
+ #undef NGROUPS_MAX
+ #define NGROUPS_MAX 512
+
static void
usage(void)
{
*** ./build-tree/shadow-19990827/src/useradd.c.~1~ Fri Feb 9 02:06:01 2001
--- ./build-tree/shadow-19990827/src/useradd.c Fri Feb 9 03:28:52 2001
***************
*** 53,58 ****
--- 53,61 ----
#endif
#include "faillog.h"
+ #undef NGROUPS_MAX
+ #define NGROUPS_MAX 512
+
#ifndef SKEL_DIR
#define SKEL_DIR "/etc/skel"
#endif
*** ./build-tree/shadow-19990827/src/newgrp.c.~1~ Fri Feb 9 02:06:00 2001
--- ./build-tree/shadow-19990827/src/newgrp.c Fri Feb 9 03:29:10 2001
***************
*** 49,54 ****
--- 49,57 ----
static GETGROUPS_T *grouplist;
#endif
+ #undef NGROUPS_MAX
+ #define NGROUPS_MAX 512
+
static char *Prog;
static int is_newgrp;
The sshd daemon has been rebuilt with the following patch so that CVS
ssh operations have the proper set of groups. The sources are in
/usr/local/src/openssh-1.2.3/ and the corresponding debian package is at
/usr/local/src/ssh_1.2.3-9.2loic_i386.deb. The package was tagged on
hold using dselect to prevent accidental upgrade. Note that this patch
may have hideous impact for users who have real account and use ssh
since most of the commands that deal with groups have not been
recompiled to handle more than the limit of 32 groups. For instance the
id command will core dump. Here is the patch applied on the
distribution:
*** sshd.c.~1~ Fri Mar 17 04:40:18 2000
--- sshd.c Tue Feb 13 06:32:17 2001
***************
*** 147,152 ****
--- 151,240 ----
const char *display, const char *auth_proto,
const char *auth_data, const char *ttyname);
+ #ifdef AUTH_SERVER_SUPPORT
+ #ifdef HAVE_GETSPNAM
+ #include <shadow.h>
+ #endif
+ #endif /* AUTH_SERVER_SUPPORT */
+
+ /* The GNU C Library currently has a compile-time limit on the number of
+ groups a user may be a part of, even if the underlying kernel has been
+ fixed, and so we define our own initgroups. */
+ #include <grp.h>
+ static int
+ xinitgroups (char *user, gid_t gid)
+ {
+ struct group *grp;
+ gid_t *buf;
+ int buflen, ngroups;
+
+ /* Initialise the list with the specified GID. */
+ ngroups = 0;
+ buflen = 16;
+ buf = malloc (buflen * sizeof (*buf));
+ buf[ngroups ++] = gid;
+
+ setgrent ();
+ while ((grp = getgrent ()))
+ {
+ /* Scan the member list for our user. */
+ char **p = grp->gr_mem;
+ while (*p && strcmp (*p, user))
+ p ++;
+
+ if (*p)
+ {
+ /* We found the user in this group. */
+ if (ngroups == buflen)
+ {
+ /* Enlarge the group list. */
+ buflen *= 2;
+ buf = realloc (buf, buflen * sizeof (*buf));
+ }
+
+ /* Add the group id to our list. */
+ buf[ngroups ++] = grp->gr_gid;
+ }
+ }
+ endgrent ();
+
+ /* Return whatever setgroups says. */
+ buflen = setgroups (ngroups, buf);
+ free (buf);
+ return buflen;
+ }
+ #define initgroups xinitgroups
+
+ /* This worked fine, and was adopted into glibc, until setgroups got a
+ similar limitation, so we override it as well. */
+ #include <linux/posix_types.h>
+ #include <sys/syscall.h>
+ #include <errno.h>
+
+ int
+ setgroups (size_t n, const gid_t *groups)
+ {
+ size_t i;
+ __kernel_gid_t kernel_groups[n];
+
+ for (i = 0; i < n; i ++)
+ kernel_groups[i] = groups[i];
+
+ {
+ long res;
+ __asm__ volatile ("int $0x80"
+ : "=a" (res)
+ : "0" (__NR_setgroups),"b" ((long)(n)),
+ "c" ((long)(kernel_groups)));
+
+ if ((unsigned long)(res) >= (unsigned long)(-125)) {
+ errno = -res;
+ res = -1;
+ }
+ return (int) (res);
+ }
+ }
+
/*
* Remove local Xauthority file.
*/
The cron daemon was recompiled from /usr/local/src/cvs-1.11.1p1/ with
the following patches applied.
/usr/local/src/cvs-1.11.1p1-2001-12-11.patch for NGROUPS_MAX
/usr/local/src/cvs-1.11.1p1-cvsroot.patch for -allow-root-file.
It prevents the multiplcation of -allow-root arguments for anonymous
cvs.
The cron daemon was recompiled from /usr/local/src/cron-3.0pl1/ with
the following patch applied, to fix the NGROUPS_MAX limit.
*** do_command.c.~1~ Tue Jun 12 06:35:48 2001
--- do_command.c Tue Jun 12 06:25:48 2001
***************
*** 30,35 ****
--- 30,112 ----
# include <syslog.h>
#endif
+ /* The GNU C Library currently has a compile-time limit on the number of
+ groups a user may be a part of, even if the underlying kernel has been
+ fixed, and so we define our own initgroups. */
+ #include <grp.h>
+ static int
+ xinitgroups (char *user, gid_t gid)
+ {
+ struct group *grp;
+ gid_t *buf;
+ int buflen, ngroups;
+
+ /* Initialise the list with the specified GID. */
+ ngroups = 0;
+ buflen = 16;
+ buf = malloc (buflen * sizeof (*buf));
+ buf[ngroups ++] = gid;
+
+ setgrent ();
+ while ((grp = getgrent ()))
+ {
+ /* Scan the member list for our user. */
+ char **p = grp->gr_mem;
+ while (*p && strcmp (*p, user))
+ p ++;
+
+ if (*p)
+ {
+ /* We found the user in this group. */
+ if (ngroups == buflen)
+ {
+ /* Enlarge the group list. */
+ buflen *= 2;
+ buf = realloc (buf, buflen * sizeof (*buf));
+ }
+
+ /* Add the group id to our list. */
+ buf[ngroups ++] = grp->gr_gid;
+ }
+ }
+ endgrent ();
+
+ /* Return whatever setgroups says. */
+ buflen = setgroups (ngroups, buf);
+ free (buf);
+ return buflen;
+ }
+ #define initgroups xinitgroups
+
+ /* This worked fine, and was adopted into glibc, until setgroups got a
+ similar limitation, so we override it as well. */
+ #include <linux/posix_types.h>
+ #include <sys/syscall.h>
+ #include <errno.h>
+
+ int
+ setgroups (size_t n, const gid_t *groups)
+ {
+ size_t i;
+ __kernel_gid_t kernel_groups[n];
+
+ for (i = 0; i < n; i ++)
+ kernel_groups[i] = groups[i];
+
+ {
+ long res;
+ __asm__ volatile ("int $0x80"
+ : "=a" (res)
+ : "0" (__NR_setgroups),"b" ((long)(n)),
+ "c" ((long)(kernel_groups)));
+
+ if ((unsigned long)(res) >= (unsigned long)(-125)) {
+ errno = -res;
+ res = -1;
+ }
+ return (int) (res);
+ }
+ }
static void child_process __P((entry *, user *)),
do_univ __P((user *));
***************
*** 240,246 ****
*/
setgid(e->gid);
# if defined(BSD) || defined(POSIX)
! initgroups(env_get("LOGNAME", e->envp), e->gid);
# endif
setuid(e->uid); /* we aren't root after this... */
chdir(env_get("HOME", e->envp));
--- 317,323 ----
*/
setgid(e->gid);
# if defined(BSD) || defined(POSIX)
! xinitgroups(env_get("LOGNAME", e->envp), e->gid);
# endif
setuid(e->uid); /* we aren't root after this... */
chdir(env_get("HOME", e->envp));
*** cron.c.~1~ Tue Jun 12 06:35:35 2001
--- cron.c Tue Jun 12 06:17:13 2001
***************
*** 25,35 ****
#include "cron.h"
#include <signal.h>
- #if SYS_TIME_H
- # include <sys/time.h>
- #else
# include <time.h>
- #endif
static void usage __P((void)),
--- 25,31 ----
The sftp is provided to Savannah project members and is
implemented in the following way to ensure security.
The sources of savannah-sftp-server and savannah-sftp are located in /usr/local/bin.
The files added to /upload are the following:
lib: total 2136 -rwxr-xr-x 1 root root 90210 Jul 12 11:41 ld-linux.so.2 -rwxr-xr-x 1 root root 1153784 Jul 12 11:41 libc.so.6 -rw-r--r-- 1 root root 771088 Jul 12 11:40 libcrypto.so.0.9.6 -rw-r--r-- 1 root root 8008 Jul 12 11:41 libdl.so.2 -rw-r--r-- 1 root root 69472 Jul 12 11:40 libnsl.so.1 -rw-r--r-- 1 root root 7600 Jul 12 11:39 libutil.so.1 -rw-r--r-- 1 root root 54632 Jul 12 11:39 libz.so.1 usr: total 4 drwxr-xr-x 2 root root 4096 Jul 12 12:07 lib usr/lib: total 40 -rwxr-xr-x 1 root root 14433 Jul 12 12:08 savannah-sftp-server -rwxr-xr-x 1 root root 23848 Jul 12 11:20 sftp-server
The ssh service is bound to
lsh with a fallback to ssh
for protocol version 1. The startup of lsh is done with the
/etc/init.d/lsh script.
The version of lsh installed is 1.4 compiled in
/usr/local/src/lsh-1.4. It includes a patch for dealing
with the NGROUPS_MAX problem described in another chapter. The
patch was activated with the following:
--- config.h.in.~1~ Thu May 16 17:38:54 2002
+++ config.h.in Wed Jun 5 08:01:13 2002
@@ -1,5 +1,7 @@
/* config.h.in. Generated from configure.ac by autoheader. */
+#define INITGROUPS_WORKAROUND 1
+
/* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP
systems. This function is required for `alloca.c' support on those systems.
*/
The people involved in this installation are Niels Moller (author of lsh), Gordon Matzigkeit (author of the NGROUPS_MAX patch) and Loic Dachary who did the installation.
Should a problem occur with this version of lsh, one has to send a bug
report to [email protected]Niels Moller including the
relevant /var/log/syslog lines (tagged with lshd) and a stack
trace of the core, if available. To get the stack trace do the following:
$ gdb /usr/local/sbin/lshd /core gdb> bt
With the appropriate information Niels is usually able to provide a patch within very short delays.
The menu.lst used by grub is installed at (/dev/hdb2)/boot/grub/menu.lst or, in grub jargon, hd(1,1)/boot/grub/menu.lst.
To access it
mount /dev/hdb2 /rescue edit /rescue/boot/grub/menu.lst umount /rescue
The service-entrance.gnu.org machine has two serial lines going to savannah.gnu.org. One that allows to see the console, the other that allows to power cycle the machine. More information on this subject may be found in sysadmin.texi (http://savannah.gnu.org/projects/sysadmin/).
A full Debian installation was done on /dev/hdb2 and can be used
if the installation is so corrupted that even a single boot will not
work. This emergency installation is labeled as such in the grub menu.
When booting on the emergency partition, the file systems of the regular
installation are mounted under the /subversions.gnu.org/ directory.
The grub menu file (menu.lst) is located on this partition, as explained above.
The kernel was rebuilt in /usr/src/kernel-source-2.2.19pre17-2.2.19pre17
and installed from /usr/src/kernel-image-2.2.19pre17_512_i386.deb.
It was recompiled with the following patch applied to raise the maximum
number of groups per process to 512.
*** include/asm-i386/param.h.~1~ Tue Aug 1 11:08:17 1995 --- include/asm-i386/param.h Sat May 26 15:44:10 2001 *************** *** 8,14 **** #define EXEC_PAGESIZE 4096 #ifndef NGROUPS ! #define NGROUPS 32 #endif #ifndef NOGROUP --- 8,14 ---- #define EXEC_PAGESIZE 4096 #ifndef NGROUPS ! #define NGROUPS 512 #endif #ifndef NOGROUP *** include/linux/limits.h.~1~ Tue Dec 2 16:44:40 1997 --- include/linux/limits.h Sat May 26 13:47:52 2001 *************** *** 3,9 **** #define NR_OPEN 1024 ! #define NGROUPS_MAX 32 /* supplemental group IDs are available */ #define ARG_MAX 131072 /* # bytes of args + environ for exec() */ #define CHILD_MAX 999 /* no limit :-) */ #define OPEN_MAX 256 /* # open files a process may have */ --- 3,9 ---- #define NR_OPEN 1024 ! #define NGROUPS_MAX 512 /* supplemental group IDs are available */ #define ARG_MAX 131072 /* # bytes of args + environ for exec() */ #define CHILD_MAX 999 /* no limit :-) */ #define OPEN_MAX 256 /* # open files a process may have */
The configuration of the IDE disks are done in /etc/init.d/hdparm.
That boosts the transfer rate from 4.4Mb/s to 23.4Mb/s.
hdparm -k 0 -d 1 -c 3 -m 16 -a 16 -u 1 -X66 /dev/hda hdparm -k 0 -d 1 -c 3 -m 16 -a 16 -u 1 -X66 /dev/hdb hdparm -k 0 -d 1 -c 3 -m 16 -a 16 -u 1 -X66 /dev/hdc
The theme system is simpler than the SourceForge original one : HTML code isn't redifined, the only differencies between themes are CSS file and images.
To create, add a theme, a new CSS file, taking as template savannah.css, should be aded in www/css with a filename that would be the theme name (as new_theme.css).
Also, the set of images should be added in www/images/new_theme.theme/* . To get the classical images with transparent background, just copy the files in www/images/transparent.themes. It is recommanded to add a non-transparent background to images since only decent www browser support well PNG transparency (ie, today, the 19 april 2002, only Konqueror and Mozilla support it).
Basically, it works by setting a cookie called SV_THEME (and eventually a SV_THEME_RANDOM) cookies. See www/include/theme2.php
The file savannah.el can be used to get shortcuts to frequently
moderation-related commands.
You should add this file to your GNU Emacs elisp path and add the
following to your .emacs
(require `savannah)
;; or
(autoload 'savannah "/home/vrac/savannah/gnuscripts/savannah.el")
There are many functions that insert canned answers for project
moderation purpose. It's usefull for gnus or VM users. Every functions
begin with sv. So you just have to
type sv-[TAB] to get a list of commands available.
# The license stuff is missing in the tarball
# It uses open in the name
# There's confusion between commercial and proprietary
M-x sv-[TAB]
M-x sv-no-license
M-x sv-use-open-in-name
M-x sv-confusion-commercial-and-proprietary
M-x sv-end-message
Additionally, there are canned bash commands available. They begin with
sv-term. You need to run them in a shell/eshell. For instance:
# Logging in
M-x eshell
$ ssh [email protected]
[...]
# I want the list of projects waiting for approval.
M-x sv-term-project-pending-list
[RET]
# This project cannot be hosted here
# This function will remove the project from the database
# and add a log message using sv_register_discard
M-x sv-term-project-registration-discard
[RET]
# A user need to be renamed
M-x sv-term-user-rename
# A user need to be removed
M-x sv-term-user-remove
You're welcome to add/enhance this script. Keep in mind it should make use of preexisting non-elisp scripts as possible and try to conform to a coherent syntax (sv-use if a project use something and it's not ok, sv-no if something is missing, sv-term for terminal-related stuff, sv-term-user of for user-related stuff et caetera)