HTTPDIST

October 1, 1999

This is the first pre-release of httpdist.

How to get software releases and updates using httpdist

With httpdist-based software distribution, all you need is one HTTP-based URL, pointing to a packing list file. This text file lists all the files in the distribution in a format compatible with the httpsync standard. To get the software, create an empty directory, change to it, and run the command:

	httpdist @<URL>

A typical session transcript will be:

	$ httpdist @https://www.equi4.com/httpdist/Packing.list
	 0 matches, 2 additions, 0 changes, 0 deletions
	Apply these changes to C:/tmp/httpdist ? y
	  ./Package.info
	a ./httpdist.tcl
	a ./index.html
	httpdist: Software packing lists, archives, and distribution
	Oct 1, 1999: internal pre-release
	See the "index.html" file for details
	$

All files will be fetched, using the standard HTTP protocol. Whenever you want to update these files you can repeat the above, or simply type:

	httpdist @

This will compare the current files with the most recent packing list, and only fetch missing and updated files.

WARNING: When you accept the update, obsolete and unknown files will be deleted!

How to get httpdist

There is a chicken and egg problem here. Httpdist itself is distributed and updated with this same mechanism, which is great once you have it, but how do you get started? There are several ways:

1. Download the httpdist.tcl script, along with a copy of TclKit, a standalone version of the Tcl/Tk scripting language which runs out of the box. TclKit includes the Trf extension which you will need to create packing lists which support message-digest archives (see below). There are builds of TclKit for Windows, Macintosh, and several Unix systems. On Windows, the command line version of TclKit is called "tclkitsh".
2. Get the httpdist.tcl script, and a suitable version of Tcl/Tk. The script runs on any platform for which there is a Tcl/Tk installation (version 8.0 or newer). On Unix, chances are that you have Tcl/Tk already installed, as a command called tclsh (or tclsh8.0, or tclsh8.2). There are many releases at the Scriptics site. Or you can download the full source code and compile it yourself.
3. Use the "httpsync" utility from Mib Software. This works because httpdist uses a compatible format for the packing list. Httpsync does not offer the other functionality of httpdist, described below.

On Windows, create a batch file called "httpdist.bat", containing a single line (add paths as needed):

	@tclkitsh httpdist.tcl %1 %2 %3 %4 %5 %6 %7 %8 %9

On Unix, if you have "tclkitsh" installed, simply rename "httpdist.tcl" to "httpdist", make the file executable, and place it somewhere on your execute path.

How to create software releases

This is one area where httpdist makes things simple. Say you have a directory called "mydist" containing all files you want to release. First, you create the "packing list":

	$ cd mydist
	$ httpdist pack
	 File 'Packing.list' created (2 files, total 27 Kb)
	$

Check the packing list, and make sure the files listed are the ones you want to include. To exclude certain files, you can list them as extra arguments (the default is to exclude "CVS" and "core"). When ready to distribute, you need to create a directory on the server - and you must have FTP access to it. Httpdist sends out all files with FTP, and fetches updates using HTTP:

	$ httpdist send ftp://[email protected]/my/package/dir
	Password: xxx
	 0 matches, 2 additions, 0 changes, 0 deletions
	* ./Packing.list
	a ./httpdist.tcl
	a ./index.html
	* ./Package.info
	$

You will be asked to enter your FTP password (in cleartext for now, sorry). Note that the packing list and the package info are always sent. When you want to send updates, re-create the packing list, and then repeat the send command. Send settings are saved, you can later repeat the above send as:

	httpdist send mysite.com

You can send the same package to multiple sites, by issueing several send commands.

How to archive obsolete files

Httpdist can build up an archive of all older files you have ever released. To do this, specify an additional directory on the FTP server where httpdist should place obsolete files. This path should be relative to the distribution path:

	httpdist send ftp://[email protected]/my/package ../../archive

When send is about to overwrite or delete a file on the FTP server, it will now instead move the file out of the way. The name used for the file is based on the MD5 message digest, derived from the contents of the file (almost like a unique checksum). These message digest values are listed in the package lists, and most importantly, the package lists themselves are also archived.

As a consequence, the FTP server will never delete a file, unless it already had a copy of it anyway.

Note that settings are remembered for each send destination, so the next time you run "httpdist mysite.com", the same settings will be used as before (user, password, directory, and archive path). These settings are saved in a file called "Packurl.send" (which is never sent out).

The "Package.info" file

This is another file which participates in the distribution process. Httpdist will always append a log of send actions to it, but apart from that its contents is free for any purpose. As maintainer, you are free to remove or reformat the log and even to delete the file, but you cannot prevent new entries from being appended. The package info file plays a crucial role in the automated archive mechanism.

A convenient feature of httpdist is that it will display the first few lines of this "Package.info" file after every update, making this is a good place to store a one-line description, a news headline, and a copyright notice, for example.

Note: the package info file is sent out and updated on every run, even though it is not listed in the packing list. You do not need to re-build the packing list before a send if you only edited the package info file.

What information goes where, then?

It's very simple:

• Put one to three lines of info in the "Package.info" file, describing the package. Keep this text up to date, as it is shown to users on every update they perform.
• Put everything else in your own files, e.g. "README", "index.html", etc.

Do not ever edit the "Packing.list" - always create it through "httpdist pack". Once the packing list has been created, do not alter anything except the package info file.

Archiving strategies

Archiving is one of the time-saving features of httpdist. With archiving, you never need to worry about losing any data ever again, or not being able to return to a previous state. There is only one drawback... you will need disk space to store all obsolete files. Then again, only obsolete files are archived - and only once, no matter how often you send out updates.

One approach which works well, is to use two different FTP servers. One locally (it could even be the same machine), with archiving turned on. The other is the public FTP server where you will allow people to fetch the latest release from. The basic cycle then becomes:

• Get that next software release done
• httpdist pack
• Verify that the packing list is what you want it to be
• httpdist send ftp://me@mylocalsite/packagedir ../archive
• httpdist send ftp://[email protected]/package/dir

In other words: always first send the files out to your archive site, and then to your public site. You can also archive more intermediate versions than the ones sent out in public, by not always performing that second send.

Retrieving old releases from the archive

This part of httpdist has not yet been completed (hey, it's only pre-release!).

More features...

There is much more one could do with httpdist, although the current release is quite effective as is. One idea is to add a "test" option, to test the integrity and report the status of the current area (both on the receiving and on the "publishing" side).

-- Coen Siegerink <[email protected]>