[Starkit] Improving Tcl compression and performance

[Guaca Mole]

> >
> > If I pack it in the starkit it takes 50 minutes
> and
> > its size is 405Mb
> 
> Not bad!

Yes, but not enough for my needs. I want tclkit to
replace distribution of potentially .tar.gz files,
cant do that if resulting archives is double size :(

> You're comparing C code versus scripted code and
> single compression 
> versus per-file compression.

Yes, just looking for ways of improving the tclkit
side of things
 
> It's unlikely that speed can be improved
> substantially without a major 
> rewrite of the Mk4vfs driver and the compression
> logic used (there's 
> more copying going on than strictly needed).  I'd be
> interested to hear 
> about acces/unpack performance comparisons, if you
> have any.

I havent timed it, but unpacking time is reasonable

> W.r.t size the comparison is not quite fair:
> starkits (and zip 
> archives) support instant access to individual
> files, tar/gzip does 
> not.  

The idea is to use starkit for selfcontained
distribution of files as opposed to .tar.gz 
This works ok for small number of files, but
differences are just way too big when number of files
increases (for the reasons you mention)
 
> Having said that, it may well be that your starkit
> contains free space 
> from multiple commits made while it is being
> created.  I've just added 
> a new subcommand to "sdx" to pack any Metakit
> datafile optimally (hence 
> also starkits/starpacks):
> 	sdx mkpack <infile> <outfile>
> Only light testing so far, but it's relatively
> simple code so it should 
> be ok.

It helps a little bit. Just tried it on a 
26.9Mb file and went down to 26.1Mb

Cheers



__________________________________
Do you Yahoo!?
Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
http://hotjobs.sweepstakes.yahoo.com/signingbonus