MetaKit Revision History

Newer changes at: https://www.equi4.com/metakit/CHANGES

September 1999 - MetaKit is undergoing fundamental changes at the moment. The major goal is to get rid of the "attached vs. unattached view dichotomy", and it looks like this and many other plans are coming nicely within reach. Because of these far reaching changes, the latest bug fixes will only be provided as source code (mk-1999-09-16.zip, in the registered user area), until everything has been fully implemented and tested.

mk4tcl - Prevent a crash when trying to access subviews in non-existent parents, by always returning an empty unattached subview. This version has tentatively been called 1.1.1 - 1999/09/16
Fixed a problem with view deletion. GetAs removed the view, then added it back in because it has to return a view, even in deletions. Changed to return an empty unattached view instead - 1999/09/16
Added c4_View::RelocateRows to move rows from one view to another. This is only allowed on two views (possibly the same) attached to the same storage. They must have exactly the same sub-structure, but need not be at the same level. This new feature allows efficient implementation of nested-structure rebalancing without having to copy memo fields or subviews - 1999/09/04
Changed c4_View::Commit and Rollback to return false instead of crashing, when called on an in-memory storage (i.e. opened without file). Both of these are only valid in combination with file-based storage objects, use LoadFromStream/SaveToStream otherwise - 1999/09/03
Added c4_MemoRef::Access/Modify calls to efficiently use memo fields. These calls provide partial access (without buffer copying for high-performance reads), and allow efficient insertion/deletion/resizing at any offset of a memo value. A potential problem with repeated memo-field resizing has also been uncovered and fixed with this change - 1999/09/01
Added a new c4_Reference::GetSize call, returns item size in bytes, except that adaptive ints return <0 if they are 1/2/4 bits, and subview "size" is the number of rows - 1999/08/31
Optimized to avoid copying of string/bytes/memo fields which have not been modified (almost all cases now return a pointer into the memory mapped file, regardless of size) - 1999/08/31
Fixed bug which prevented memo properties from being serialized properly (c4_Storage::StoreToStream/LoadfromStream). A new regression test (s33) verifies that this now works - 1999/08/28
Fixed serious bug with memo properties: when a memo value contains data and is then set to empty, an assertion is triggered in the debug build, but the release could crash or even damage the datafile. A new regression test (s32) verifies that this now works - 1999/08/28
Added c4_Strategy::_keepAllColGaps (defaults to false) to optionally keep all gaps in modified columns until committed. The default is to cache the 50 most recently modified columns, which is slower (especially for views with >50 properties) - 1999/08/28
Added c4_Strategy::_clearOnDetach (defaults to false) to optionally prevent views from being copied when their storage object goes away before they do. Views pointing into the storage will be resized to zero rows (but derived views may end up being incorrect). This flag can be used to prevent copying large amounts of data when the order of object destruction cannot be controlled - 1999/08/28
mk4tcl - Changed fdopen to _fdopen to run correctly (why?) with Tcl 8.2 (and probably 8.1) - 1999/08/26
Added c4_Strategy::IsValid() to allow checking for open success - 1999/08/24
Prevent crash when GetAs is used to delete a non-existent view - 1999/08/24
Fixed bug on Windows with very complex structure defs (over 4 Kb) - 1999/07/17

Mk4tcl 1.1 released on 2-JUN-1999

mk4tcl - The call "mk::file open" (no further args) now returns a list with tag- and file-names of all open files - 1999/06/01
mk4tcl - Added capability to use in-memory databases (omit file name or use an empty string in "mk::file open") - 1999/05/28
mk4tcl - Fixed two bugs in "mk::loop" (failing to dup a shared loop variable, and break/error handling) - 1999/05/27

MetaKit 1.8.6 released on 26-MAY-1999

mk4tcl - Don't resize view if "mk::row append" or "mk::set" fails - 1999/04/08
tclkit - Rebuilt for Linux, Windows, and Solaris with selection and locking ok - 1999/04/08
mk4tcl - At last the locking mechanism on Solaris seems to work. It uses the /usr/ucb* compatibility layer for flock advisory locking - 1999/04/06
mk4tcl - Fixed a nasty select bug introduced on 3/18 - 1999/04/06
Made all derived c4_Property class members const: Get, Set, operator [], and AsRow - 1999/03/18
mk4tcl - Fixed bug which prevented multiple-min and multiple-max selections from working correctly (ouch) - 1999/03/18
mk4tcl - Renamed to "Mk4tcl" (was "mk4tcl", i.e. lower case). You must now specify "package require Mk4tcl 1.1" to load the new release - 1999/03/17
mk4tcl - Added real locking to Mk4tcl, so that files open for writing cannot be opened more than once (one writer *or* multiple readers, no mix). This removes the need to use lock files, and is particularly useful for "scripted documents". Some weaknesses remain: Solaris does not prevent reopening inside the same process, and locking may not always work properly with files via NFS - 1999/03/17
Fixed double alignment sensitivity w.r.t. Ultrasparc in c4_Bytes buffer - 1999/03/16
Commit logic extended to better deal with write errors (such as disk full). In such cases, commit is cancelled, and now also returns false (API change) - 1999/02/28
mk4tcl - Changed to throw an exception on commit failures - 1999/02/28
Changed some logic in the "mk4dll.h" to avoid "__declspec(dllexport)" warnings in MSVC - 1999/02/22
mk4tcl - Memory leak in "mk::select" fixed - 1999/02/18
mk4tcl - Made changes to allow using datafiles which start off with a non-MetaKit header. Mk4tcl now skips anything up to the magic number of a MetaKit datafile, which must be on a 16-byte boundary, and within the first 4 Kb of the file. This is for use with the new "scripted documents" and TclKit - 1999/02/18
Fixed a serious problem with memory mapped filed under Windows 95/98, when the datafile is on a network server (Samba, in my case). This fix is not needed for NT - 1999/02/13

MetaKit 1.8.5 released on 13-JAN-1999

Minor source code changes (constructor initialization, mostly), to make sources compile without any warnings on egcs-2.91.60 (1.1.1), even with "-fguiding-decls -Wall -pedantic -Wno-unused" - 1999/01/12
Switched to new libtool 1.2e (from its official CVS tree, on 9-Jan-1999), which supports a few more more platforms and has bugfixes - 1999/01/09
Bumped the copyright year to 1999 in the MetaKit source files - 1999/01/09
Fixed a bug in c4_View::ProjectWithout, the new property count was not reduced. This also affected c4_View::Unique - 1998/12/28
mk4py - A few more fixes, basic code works. Still not robust enough with bad args. No selection code yet, other than "find" and "match" (regexp on one string field) - 1998/12/27
Added Henry Spencer's regular expression matching package (regexp only, no regsub), accessible through new c4_View::Match() member. Implemented in a single source file, "src/regexp.cpp". Only for string properties - 1998/12/23
Removed all the last references to c4_String in the "mk4.h" header. The string class is now available by including "mk4str.h". Also removed all the last traces of q4_MFC/q4_STD/q4_UNIV settings. This means that the same library build can now be used in more configurations (major internal change, affects c4_String uses) - 1998/12/14
Switch to stdio package for MFC builds. This simplifies code and unifies the strategy routine, but also means a CFile object can no longer be used for IO with the standard strategy class. Could still create a derived class to work with CFile, though (major internal change, affects the MFC version) - 1998/12/14
Dropped c4_String use in c4_StringArray for the Universal build (new bug, fixed) - 1998/12/14
mk4py - Fixed property setting code to always add new properties - 1998/12/14
mk4py - Accept anything with a "__dict__" attribute, such as class instances, as input for initializing a row - 1998/12/14
mk4py - Disambiguate access to properties and member funtions by stripping a leading underscore in property lookup. This prevented the use of member functions on cursors and storage objects - 1998/12/09
mk4py - Cursor and storage objects now also behave like dictionaries: has_key(x), keys(), values(), and items() - 1998/12/09
mk4py - Datafiles are now commited on close by default (mode = 1). Open mode: -1 = write/no-commit, 0 = read-only, 1 = write/autocommit/default - 1998/12/09
mk4py - Layout description fixed, working around a c4_View::Describe flaw - 1998/12/09
mk4tcl - More invalidation after restructuring to force re-attachment of views - 1998/12/07
Univ and STL builds did not cleanup the column cache in ~c4_Strategy. Fixed - 1998/12/03
mk4tcl - Lots of problems with shared lib linkage on Linux. Solved by going back to the standard 2.7.2.3 version of gcc which comes with RedHat 5.0 (the build command now used is "gcc -shared -o mk4tcl.so mk4tcl.cpp -Wl,../mk/unix/solib.a" - no "-ltcl", this now uses back-linking) - 1998/11/24
Removed the MSVC-/MFC-specific memory allocation checks. These were not compatible with BoundsChecker and Purify anyway. Only affects debug assertion checking - 1998/11/24
mk4tcl - 1.0.6 - Fixed a bug in sorting numeric data (and strings across segment boundaries), introduced by new optimized code. Solved by creating a buffer pool in the c4_Persist class and rotating on each new request. A new test (s31) verifies proper operation. This only affected Mk4tcl, which sorts and used the new code - 1998/11/04
Removed reference to "k4dll.h" in src/header.h, exports are now set from mk4.h header. Also switched to export by name for MSVC builds of DLL, maintenance became too painfull - 1998/11/04
Fixed a 8-byte alignment problem on OSF and HP-UX by reordering c4_Bytes fields - 1998/11/04
Alpha does not use longs under Win NT, adjusted test of q4_LONG64 accordingly - 1998/11/04
Fixed a bug in c4_View::GroupBy, and added new test (c18) to verify proper operation. Does not affect Mk4tcl or Mk4py, which do not use GroupBy - 1998/10/14
mk4tcl - 1.0.5 - uses new fast lib, changed from Universal to faster MFC-based build under Windows - 1998/10/09
mk4tcl - Return an error when datafile creation failed, it used to crash - 1998/10/09
Major string access performance optimization, as well as a first go at using a fast property lookup map. The "pStringProp (row)" operation now returns a (const char*) instead of a c4_String, this is likely to affect user code in various ways - the general advice is to remove as many intermediate "c4_String" casts and objects and possible. As a result, string access may end up more than 5x faster (major change) - 1998/10/08
Version number in "mk4.h" header of MetaKit 1.8.4 still says "183" (will be fixed in the next release) - 1998/10/05
mk4tcl - 1.0.4 - The generation counter fix of 1.0.3 was wrong for temp rows, fixed - 1998/10/01

MetaKit 1.8.4 (also Mk4tcl 1.0.3, Mk4py 0.3) - released on 26-SEP-1998

Many changes to adjust all win16 and win32 builds to the most recent header and directory structure changes. All MSVC 1.52 projects now build correctly. Also: upgraded from MSVC 5.0 to the new 6.0 release - 1998/09/26
mk4tcl - 1.0.2 - Added a generation counter to MkPath, and force a relookup of all paths whenever a datafile is closed to avoid stale views - 1998/09/23
mk4tcl - More checks to avoid crashing when a bogus view/path is specified. Path parsing now stops at the first view which is unknown or not a subview - 1998/09/23
mk4py - Allow any Python object where a row is expected (fetch attributes into a temporary row) - 1998/09/23
mk4py - Removed "SetSize(123)" test code from the "layout" member (whoops!) - 1998/09/22
mk4py - Added "view[-1]" indexing logic, i.e. relative to the end - 1998/09/21
mk4tcl - 1.0.1 - View names with underscores were not parsed correctly, fixed - 1998/09/18
mk4tcl - Prevent crash when accessing a non-existent tag, and when deleting a non-existent view - 1998/09/18

MetaKit for Tcl 1.0 for Linux, Windows, Macintosh - released on 15-SEP-1998

mk4tcl - Added net-aware installation and lookup of new platform builds. The mini installer can now connect to www.equi4.com and download new builds as they become available - 1998/09/14
mk4tcl - Changed global MkWorkspace into a ref-counted interpreter-specific heap object, with cleanup using Tcl exit handlers. Fixes cleanup problems with Linux and Mac - 1998/09/13
mk4tcl - Added -nocommit option to "mk::file open" - 1998/09/13
mk4tcl - The "demo.tcl" script now also includes an optional mini-installer - 1998/09/13
mk4tcl - Fixed compound selection/sorting bug (row map was incorrect) - at last - 1998/09/10
mk4tcl - Started writing a more extensive set of manual pages and examples - 1998/09/08
mk4tcl - No longer need to specify ":X" property types, unless adding new properties. This works, but using properties of the same name but different types might cause problems (use explicit types in this case) - 1998/09/07
mk4tcl - Fixed "mk::cursor create" and select multiple conditions bugs - 1998/09/07
mk4tcl - New test sets created, using the Tcl (all/defs/*.test) tools - 1998/09/07
mk4tcl - Added code to handle continue and break in mk::loop - 1998/09/06
mk4tcl - Do "fconfigure -translation binary" for each mk::load/save - 1998/09/06
The property symbol table no longer contains type information, which is now stored in each c4_Property object. A view may no longer contain two properties of the same name, differing only in type (major change) - 1998/09/06
Changed c4_Storage::View to make sure the requested property exists. This is needed for the new c4_Reference way of storing a c4_Property&, but it might also solve a long-standing problem with View vs. Store - 1998/09/06
Changed internal string buffering to include the trailing null byte. This is to prepare for more efficient string passing (moderate change) - 1998/09/03

New log format - after MetaKit 1.8.3, mk4tcl 0.7, and mk4py 0.2

MetaKit for Python, mk4py 0.2, was released on July 13, 1998 - no detailed log available.

MetaKit 1.8.3 released on 28-MAY-1998

The c4_Strategy::ResetFileMapping member has been made virtual
Fixed a bug in c4_View::RemapWith (needed for sorting in mk4tcl)
Added "mk::select" to mk4tcl - it selects, searches, and sorts
Changed streaming ("mk::file load/save") to use Tcl binary channels
On Unix, mk4tcl requires mk4.so - on Windows it's a single DLL

MetaKit 1.8.2 released on 10-APR-1998

All versions are now generated from a Linux version
The Windows and Macintosh releases remain 100% supported
This is merely a way to better automate the release process
The core header files have been merged into one: mk4.h
On Unix, the library builds are now called "libmk4.a" and "libmk4.so"
Started using GNU autoconf, see the "unix" subdirectory
All Windows-specific files have been moved to the "win" directory
All Macintosh-specific files have been moved to the "mac" directory

MetaKit 1.8.1 released on 20-MAR-1998

Fixed a machine-dependent structure alignment problem in c4_Bytes
Because of this, headers are not compatible between 1.8 and 1.8.1
Some conditional code was added to build on SunOS and Solaris

MetaKit 1.8 released on 6-MAR-1998

Added a new datatype (c4_MemoProp) for large binary objects
Added several view operations, such as Slice, RemapWith, and Rename
Added a wide range of relational operations, including Join and GroupBy
Added standard set operations: Union, Intersect, Different, and Minus
Custom read-only views, including a generic DBASE file reader as example
Added a datafile converter/optimizer, with source code (MkOptim)
Views can now be compared, based on lexicographic row-by-row ordering
Comparison of subviews now also uses this ordering (was a ptr comparison)
Fixed a property problem in c4_View::Project and c4_View::ProjectWithout
Fixed problem when accessing some deeply nested derived properties
Changed to a more readable class documentation format in HTML
Added a "Getting Started" document to help with setup and initial use

MetaKit 1.7 released on 28-MAY-1997

Memory-mapped files implementation added for Win32 and Unix
Major redesign and optimization of the internal c4_Column class
Removed several internal module dependencies, moved some code
Current version is up to ten times as fast as before (MSVC, Win32)
Floating point code now disabled in "small model" (Intel) builds
Added member function equivalents for operators (Get, Set, AsRow)
Dramatic performance increase for string access in large views
Now avoids fragmentation during save, so loading will be faster
Fixed a resizing problem with doubles in nested sub-views
Compiles/tests ok on Mac (CW11/68k), w/member ptr workaround
Changed binary comma operator in c4_View to allow use with STL
Added "KitBinder" demo to embed MetaKit data in an executable
Complete redesign of the documentation, switched to "Perceps"
Added STL-based interface (w/stdio) in the Universal release
The c4_Field and c4_Table classes/headers are no longer public

MetaKit 1.62 released on 18-MAY-1997

Fixed a problem with bit-field resizing (1/2/4 bits to 16/32)
On-the-fly restructuring of the new binary properties didn't work
Got rid of the static property limitations of 1.6 under Win32

MetaKit 1.61 released on 5-MAY-1997

Added MSVC 5.0 support: both 4.2 and 5.0 are now in MSVC32 dir
Minor source adjustments (MSVC 5 bool, c4_BytesHandler::ulong)
Removed duplicate code in auto-sizing int code, now faster

MetaKit 1.6 released on 29-APR-1997

Added binary properties, based on the existing c4_Bytes class
Thread-safe extensions ("STA model") for ActiveX use (Win32)
Important: string comparisons in views are now case-insensitive
The c4_View::Search member now does a normal binary search
Small integers are automatically packed in 1/2/4 bit fields
Can release all memory on commit (this is now the default)
Added c4_Storage::AutoCommit, will force a Commit in destructor
Improved buffering when working mith multiple storage objects
Semantics of c4_View::InsertAt changed, now copies new properties
Fixed silly (but major) mistake in the c4_View::SetAtGrow member
Fixed problem with strings on 4K boundary if >32 Kb (16b only)
Sorting no longer uses globals - fixes an interaction problem
Fixed problem in change propagation with multiple derived views
Changed memcpy to memmove, fixes bug on Linux and Mac ports
Fixed free-store mixup when using extremely complex views on file

MetaKit 1.5 released on 9-DEC-1996

Added an efficient implementation of reference-counted strings
Basic collection and file (stdio) replacements implemented
Configuration headers reorganized a bit for more flexibility
Ported library and samples to Mac (Metrowerks) and Unix (gcc)
Added standalone libs for DOS and Windows, MFC now optional
Installed / moved the MetaKit project to use source code control
Double-precision storage didn't work in release 1.4, now fixed
New properties now pass through Sort/Select and ProjectWithout
Split more header files, started work on streams + STL version
Solved a problem with projection in compound derived views
In specific cases, multiple commits messed up free space, fixed
Read-only use of data files now allows sharing (but no writes)
Better change propagation in sort/select views, major speedups
Added several tests for some more complex derived view cases
Comparison of generic c4_Reference and c4_Bytes objects added

MetaKit 1.4i released for INTERNAL USE only

Version 1.4 sources passes tests on Macintosh (Metrowerks 10)
Also passes regression tests on Linux 1.2.13 (GNU gcc 2.7.2)
Required some minor source syntax tweaks for Borland 5.01 (MFC4)
Succesfully ran DEMO sample code in Optima C++ 1.0 (Universal)
The 1.4 sources work with Symantec 7.2 and Watcom 10.6 (MFC4)

MetaKit 1.4 released on 1-NOV-1996

Can use foreign files (flipped bytes), on-demand & streamed
Fixed bug with empty strings in c4_FormatS (see s16/s17 tests)
Implemented segmented vectors, dropped oversized vector alloc
Most size limitations gone on 16-bit platforms (DOS: mem limit)
Automatic storage descriptions now used by c4_Storage::Set
Many major performance improvements, fast fill/store of views
Automatic cleanup of prop-cache to avoid mem leaks (debug vsn)
Added copy constructor and assignment to the property classes
Improved loading and saving of complex data (over 500 subviews)
New c4_Strategy class encapsulates system I/O and customization
Added missing insertion code for InsertAt(int, const c4_View&)
Avoided MFC assertion for views which outlive empty storage obj
Fixed initialize of temp storage with no associated data file
Changed to new MSVC 4.2b compiler and IDE, no problems found
Compiled DEMO16 and STRUCT as a small-model MS-DOS executable
Restructure sometimes affected previously attached views, fixed

MetaKit 1.3 released on 4-OCT-1996

Fixed problem with static initializers (Win32 DLL version only)
Fixed assertion when deleting a structured subview on file
Add support for double-precision floating point (c4_DoubleProp)
Added a "DEMO" project illustrating the core MetaKit features
New "MYIO" example demonstrates use of encrypted file storage
Substantial performance improvements for string storage
Made Win32 version reentrant by using some thread-local storage

MetaKit 1.2 released on 28-MAY-1996

Catfish 1.2 adds search / sort and many small improvements
Added change propagation for selection and projection
Fixed M4KIT.WRI and M4KITAPI.HLP for use with the Win 3.1x apps
Linker problems 16-bit debug DLL gone, ok to add to reg'd vsn
Implemented c4_Table::Restructure and c4_Storage::Define
String handler uses no storage until non-empty string is stored
Derived sequence code moved to new DERIVED.CPP source file
Added FilterSeq notification, SortSeq cannot pass it on yet
Win32 DLLs now export all functions by ordinal, like Win16
Fix s960424a: absolute paths in makefiles for src distrib
CatFish 1.12 - fixed a Windows 3.1x compatibility problem
Source code released, with makefiles and test program w/output
Changed bool typedef to a #define to avoid MSVC 4.1 warnings

MetaKit 1.1 released on 19-APR-1996

Internationalization and Win95 LFN support added to CatFish
Added CATFISH sample application with full source code
Decided to keep explicit file format strings in this version
Fixed sorting/selection when used with nested views
Rollback now ok, but must be careful with existing views
Optimized c4_Bytes, now uses a local buffer for small items
Automatically increases int sizes as needed, 0/1/2/4 bytes
Simple find and search, careful which props are matched
More members documented in help file, still far from done
Removed assertions from inlined code, seemed like overkill
Added FTPCAT sample, a utility to make ftp server catalogs
Simplified c4_Persistent class, no more static _Load/_Save
In DISCAT, fScanDirectories changed to return a c4_View
Shareware registrations now also through Kagi and PsL
Uploads to ftp sites and CIS, much more info on home web site

MetaKit 1.0 beta 3 released on 12-MAR-1996

SaveNow/LoadNow added, cleaned up persistent classes
Tested several persistent combinations and multi-storage
String properties in tables now use offset index for speed
Data handler classes moved to new HANDLER.CPP source file
Removed 64 Kb column size limits in 32-bit vsn (ulong's)
Seq's grab their data when underlying tables are destroyed
Added Project and ProjectWithout members for projections
Simplified CONFIG.H, removed #ifndef q4_... stuff
Hdrs k4table.h and k4field.h no longer included by default
Renamed SortUpDown to SortOnReverse
c4_Table::SubEntries moved from K4TABLE.INL to TABLE.CPP

Start of change log