[Metakit] Corrupted database over network - WinNT

[Riccardo Cohen]

Hi Jean-Claude,
This is very clear, thank you. I really hope that vista corrected this windows bug. It is 
not the only bug with windows networking, but this one is a real pain.
The problem now for us at least, is that we have many different way of using metakit. 
sometimes we encapsulated in one of our class, wich make it easy to redefine a file 
strategy, but sometimes we access directly for performance reason, and there are many 
places where we open a storage.

Maybe I'm wrong, but there might be a very easy and effortless solution (for developers 
that never have 2 C4_storage on the same file) :

in c4_FileStrategy::DataOpen(), I tried to replace :

fd = _wopen(wName, flags);

by :

fd=_wsopen(wName,flags,_SH_DENYRW);

Unfortunately it did not work, but really if someone know something about windows open() 
function, this could help many people.

Thanks for any clue :)



Jean-Claude Wippler wrote:
> Thanks, Pat Thoyts for your detailed analysis and description of the 
> workaround you found.
> 
> Let me summarize as follows:
> 
>> Metakit maps the file as
>> a read-only memory mapping and when it saves it re-serializes into the
>> underlying file. It then drops the current mapping and re-maps the
>> file. What I observed was that when it re-mapped it got the original
>> data and did not in fact re-map the newly written data at all.
> 
> In other words, Windows does not guarantee a consistent view of the data 
> it buffers for networked file systems: write something to a file, map it 
> to memory *afterwards*, and apparently you can get stale data.
> 
> If MK were to use r/w maps and push its data through there, then 
> presumably it would work.  That will not work in the case when the file 
> has to be extended, though.  Keep in mind that MK can write data to file 
> for which partially comes from the r/o map.  Furthermore, MK maps r/o 
> deliberately, because it prevents stray pointer writes from damaging a 
> MK datafile.
> 
> So what we have today, is a design which works on Windows local disks, 
> and on Unix/MacOSX both locally and on remote file systems (both NFS and 
> SMB afaik).
> 
> [From: <Hector at coollector.com>]
>> 1/ You confirm that metakit files have a serious design flaw that puts 
>> at risk our data when accessed through a network.
> 
> With all due respect, the "serious design flaw" is in Windows.  Getting 
> that fixed is beyond our control.  We can only hope that XP or Vista get 
> it right.
> 
>> 2/ You give indications on the nature of the corruption, and how to 
>> prevent it.
> 
> Yes, and for that I am very grateful.  In addition to Pat's most recent 
> post, more information can be found at 
> http://www.equi4.com/pipermail/metakit/2005-August/002112.html
> 
> So the solution is to derive a c4_Strategy class which opens the file in 
> exclusive mode.  Pat has most generously handed everyone the solution on 
> a silver platter.  If you don't want to subclass c4_Strategy, then open 
> the file yourself and pass it to MK.  If you never need concurrent 
> readers, use CreateFile and always open with dwAccessMode set to 0, then 
> turn it into a FILE* with freopen and pass it to MK.
> 
> I for one never use MK in r/w mode on remote file systems.  Not just due 
> to the above risks, but because locking also leads to all sorts of 
> "Heisenberg" trouble.  In my book, shared data use should use 
> client/server solutions.  That need only take a few dozen lines of code, 
> depending on your tool choices.
> 
> -jcw
> 
> 
> _____________________________________________
> Metakit mailing list  -  Metakit at equi4.com
> http://www.equi4.com/mailman/listinfo/metakit
> 

-- 
Très cordialement,

Riccardo Cohen
ligne directe : 02-47-49-63-24
-------------------------------------------
Articque
http://www.articque.com
Moulin des Roches
37230 Fondettes - France
tel : 02-47-49-90-49
fax : 02-47-49-91-49